The advice I've got says that trying to spray an organic "insecticide" isn't worth it. So I'll take the hit to my yield.
Armyworms have earned their name for their tendency, at high populations, to ‘march’ across a field, consuming whatever vegetation is in their path. Two species of armyworm, true armyworms and fall armyworms, may affect crops in the Upper Midwest, predominantly in the grass family (corn, wheat, other small grains) and in fall grassy cover crops.
Early summer crop damage is caused by the true armyworm. True armyworm larva can cause damage in the early summer to corn, wheat, barley, oats, and occasionally to soybeans and sunflowers. The adults begin arriving in the upper Midwest in April or early May, with peak moth flights later in May or early June. These dates vary from year to year, based on weather conditions.
The link from Albert Lea Seeds has plenty of more info on them.