Survey and Statistical Analysis Methods
A. Overview
The key points for the survey design and sampling and statistical analysis for this 2016/2017 Harvest Report are as follows:
Following the methodology developed for the 2015/2016 Harvest Survey, the samples were proportionately stratified according to Agricultural Statistical Districts (ASDs) across nine key sorghum-producing states, which represented more than 99% of U.S. sorghum exports. Additionally, the samples were classified according to two Harvest Areas – Early Harvest and Late Harvest.
A total of 250 samples collected from the nine states were targeted to achieve a maximum ±10% relative margin of error (Relative ME) at the 95% confidence level for the quality factors.
There was a total of 254 unblended sorghum samples tested. These samples, pulled from inbound farm-originated trucks at local elevators, were received from July 18, 2016 through November 30, 2016.
A proportionate stratified sampling technique was used for the mycotoxin testing across the ASDs in the nine states surveyed for the other quality factors. This sampling resulted in 75 samples being tested for aflatoxins and DON.
Weighted averages and standard deviations following standard statistical techniques for proportionate stratified sampling were calculated for the U.S. Aggregate and each of the two Harvest Areas.
To evaluate the statistical validity of the samples, the Relative ME was calculated for each of the quality attributes at the U.S. Aggregate and the Harvest Areas. The Relative ME for the quality factor results was less than ±10%, except for foreign material and total damage for the U.S. Aggregate, BNFM, foreign material, and total damage for the Early Harvest Area, and total damage for the Late Harvest Area. While the lower level of precision for these quality factors is less than desired, these levels of Relative ME do not invalidate the estimates.
Two-tailed t-tests at a 95% confidence level were calculated to measure statistical differences between the 2016 and 2015 quality factor averages.
A. Survey Design and Sampling
1. Survey Design
For this 2016/2017 Harvest Report, the target population was commodity sorghum from the nine key U.S. sorghum-producing states representing more than 99% of U.S. sorghum exports. A proportionate, stratified, random sampling technique was applied to ensure a sound statistical sampling of the U.S. sorghum crop at the first stage of the market channel. Three key characteristics define the sampling technique: the stratification of the population to be sampled, the sampling proportion per stratum, and the random sample selection procedure.
Stratification involves dividing the survey population of interest into distinct, non-overlapping subpopulations called strata. For this study, the survey population was sorghum produced in areas likely to export sorghum to foreign markets. The U.S. Department of Agriculture (USDA) divides each state into several ASDs and estimates sorghum production for each ASD. The USDA sorghum production data, accompanied by USDA sorghum consumption data and foreign export estimates, were used to define the survey population in nine key sorghum-producing states representing more than 99% of U.S. sorghum exports. The ASDs were the subpopulations, or strata, used for this sorghum quality survey. From those data, the Council calculated each ASD’s proportion of the total U.S. foreign exports to determine the sampling proportion (the percent of total samples per ASD) and ultimately, the number of sorghum samples to be collected from each ASD. The number of samples collected for the 2016/2017 Harvest Report differed from ASD to ASD because of the different shares of estimated foreign export levels.
The number of samples collected was established so the Council could estimate the true averages of the various quality factors with a specific level of precision. The level of precision chosen for the 2016/2017 Harvest Report was a Relative ME of no greater than ±10%, estimated at a 95% level of confidence. A Relative ME of ±10% is a reasonable target for biological data such as these sorghum quality factors.
To determine the number of samples for the targeted Relative ME, ideally the population variance (i.e., the variability of the quality factor in the sorghum at harvest) for each of the quality factors should be used. The more variation among the levels or values of a quality factor, the more samples required to estimate the true mean within a given confidence level. In addition, the variances of the quality factors typically differ from one another. As a result, different sample sizes for each of the quality factors would be needed for the same level of precision.
Since the population variances for the quality factors evaluated for this year’s sorghum crop were not known, the variance estimates from the 2015/2016 Harvest Report were used as proxies. The variances and ultimately the estimated number of samples needed for the Relative ME of ±10% for quality factors were calculated using the 2015 results of 207 samples. Total damage, with a Relative ME of 29%, was the only quality factor for which the Relative ME exceeded ±10% for the U.S. Aggregate. Based on this outcome, the targeted sample size was increased from the 200 samples targeted in the 2015/2016 Harvest Survey to 250. It was expected that this increased sample size would allow the Council to estimate the true averages of the quality characteristics with the desired level of precision for the U.S. Aggregate, with the exception of total damage.
The same approach of proportionate stratified sampling was used for the mycotoxin testing of the sorghum samples as for the testing of the grade, moisture, chemical, and physical characteristics. In addition to using the same sampling approach, the same level of precision of a Relative ME of ±10%, estimated at a 95% level of confidence, was desired. Testing at least 63 samples (25% of the 250 targeted samples) would ensure with 95% confidence that the percent of tested samples with aflatoxin results below the U.S. Food and Drug Administration (FDA) action level of 20 parts per billion (ppb) would have a Relative ME of ±10%. It was also estimated that the percent of tested samples with DON results below the FDA advisory level of 5 parts per million (ppm) would have a Relative ME of ±10%, estimated at a 95% level of confidence. The proportionate stratified sampling approach also required testing at least one sample from each ASD in the sampling area. To meet the sampling criteria of testing 25% of the minimum number of targeted samples (250) and at least one sample from each ASD, the targeted number of samples to test for mycotoxins was 75 samples.
2. Sampling
The random selection process was implemented by soliciting local grain elevators in the nine states by email and phone. Postage-paid sample kits were mailed to elevators agreeing to provide the 2500-gram sorghum samples requested. Samples were collected from the elevators when at least 30% of the sorghum in their area had been harvested. The 30% harvest threshold was established to avoid receiving old-crop sorghum samples (as farmers cleaned out their bins for the current crop) or new crop harvested earlier than normal (for reasons such as elevator premium incentives). The individual samples were pulled from inbound farm-originated trucks when the trucks underwent the elevators’ normal testing procedures. The number of samples each elevator provided for the survey depended on the targeted number of samples needed from the ASD along with the number of elevators willing to provide samples. A maximum of twelve samples from each physical location was collected, but nearly 90% of the participating elevators submitted four or fewer samples. A total of 254 unblended sorghum samples pulled from inbound farm-originated trucks were received from local elevators from July 18, 2016 through November 30, 2016, and tested.
C. Statistical Analysis
The sample test results for the grade factors, moisture, chemical composition, and physical factors were summarized as the U.S. Aggregate and also by two groups. The groups, which harvest sorghum in differing time periods, were labeled as Harvest Areas:
The Early Harvest Area, which consists of areas that typically harvest sorghum from the beginning of July through the end of September; and
The Late Harvest Area, which consists of areas that typically harvest sorghum from the beginning of September through the end of November or later.
In analyzing the sample test results, the Council followed standard statistical techniques employed for proportionate stratified sampling, including weighted averages and standard deviations. In addition to the weighted averages and standard deviations for the U.S. Aggregate, weighted averages and standard deviations were calculated for the Harvest Areas. First, each sampled ASD was categorized by Harvest Area, based on historical USDA state-level harvest progress data, with each ASD exclusively belonging to a single Harvest Area. Second, each ASD was weighted by its estimated proportion of foreign exports. The Harvest Area and U.S. Aggregate statistics were calculated using these weights.
The Relative ME was calculated for each of the quality factors for the U.S. Aggregate and for each of the Harvest Areas. The Relative ME for the quality factor results was less than ±10%, except for BNFM in the Early Harvest Area, foreign material for the U.S. Aggregate and Early Harvest Area, and total damage for the U.S. Aggregate and both harvest areas. The Relative ME for BNFM, foreign material, and total damage are shown in the table to the right.
While the level of precision for these quality factors is lower than desired, these levels of Relative ME do not invalidate the estimates. Footnotes in the summary tables for “Grade Factors and Moisture” indicate the attributes for which the Relative ME exceeds ±10%.
References in the “Quality Test Results” section to statistical and/or significant differences between results in the 2015/2016 Harvest and Export Cargo Quality Report and the 2016/2017 Harvest Quality Report were validated by two-tailed t-tests at a 95% confidence level.