پیشرفت الگوبرداری به یک سیستم پشتیبانی تصمیم گیری برای ارزیابی تولید محصول جهانی

The Office of Global Analysis/International Production Assessment Division (OGA/IPAD) of the United States Department of Agriculture – Foreign Agricultural Service (USDA-FAS) has been assimilating new data and information products from agencies such as the National Aeronautics and Space Administration (NASA) into its operational decision support system (DSS). The FAS mission is to improve monthly estimates of global production of major agricultural commodities and provide US Government senior decision makers and the public the most accurate, timely, and objective assessment of the global food supply situation possible. These estimates are ultimately captured as the US governments’ official assessments of world food supply for the commodity markets and policy makers. The goal of this research was to measure changes in the quality and accuracy of decision support information resulting from the assimilation of new NASA products in the DSS. We gathered both qualitative and quantitative information through questionnaires and interviews to benchmark these changes. We used an interactive project lifecycle risk management tool developed for NASA mission spaceflight design and quality assurance (DDP – Defect Detection and Prevention) to do this. In this case, we used it to (1) quantify the change in DSS Objectives attained after assimilation of new products, and (2) evaluate the effectiveness of various Mitigation options against potential Risks. The change in Objectives attainment was considered the most important benchmarking indicator for examining the effectiveness of the assimilation of NASA products into OGA/IPAD’s DSS. From this research emerged a novel model for benchmarking DSSs that (1) promotes continuity and synergy within and between government agencies, (2) accommodates scientific, operational and architectural dynamics, and (3) facilitates transfer of knowledge among research, management, and decision-making agencies.

The use of Earth science data, models and geographic information systems in agricultural monitoring and assessment continues to expand our ability to understand the impacts that climate variability, landscape change, and anthropogenic and economic forces have on global agricultural production (CCSP, 2008). The most important responsibilities of the Office of Global Analysis/International Production Assessment Division (OGA/IPAD; formerly the Production Estimates and Crop Assessment Division – PECAD) of the United States Department of Agriculture – Foreign Agricultural Service (USDA-FAS; created in 1953) are producing assessments of global crop conditions and monthly estimates of planted area, yield, and production for selected commodities like soybeans, wheat, corn, rice, cotton, and oilseeds (Hammond, 1975 and Hutchinson, 1991). IPAD’s assessments are intended to promote the development of new initiatives directed at expanding US agricultural exports, combating world food insecurity, monitoring global agricultural change, and improving US crop condition and disaster assessments. IPAD’s main goal is to collect and analyze global climate, biophysical, crop, economic and field reference data, produce the most accurate production estimates and decision support information possible from these data, and then disseminate timely, objective, useful, and cost-effective global crop condition and agricultural market intelligence information with a high level of confidence in the production estimates. Low confidence forecasts can translate into more volatile markets where food shortages and over-stocks are more likely to occur. Stability of food prices requires a delicate balance between food supply and demand. Accurate assessments and forecasts of the global food supply help achieve this balance. These food supply estimates are arguably the most scrutinized and comprehensive in the world. The collaboration between USDA and the National Aeronautics and Space Administration (NASA) to improve the accuracy and timeliness of global crop production forecasts is an important component in helping to achieve market stability. This collaboration continues to introduce new NASA Earth science and measurements, model predictions, and information technology to enhance decision support for the IPAD. IPAD has developed a complex data- and model-driven decision support system that increasingly provides structure to the decision-making process and leads to more consistent and reliable crop production estimates for all countries potentially influencing US agricultural industries and policies. Enhanced decision-support derived from Earth science and the satellite perspective and based on its comprehensive view provides the objective, global, and farm-level information necessary to assess world-wide production throughout the growing season and improves the USDA’s capability to serve its management and policy responsibilities to society. As US and global socio-economic pressures increase due to globalization, population pressure, resource depletion and global climate change, IPAD analysts are facing rapidly increasing information demands. As a result, IPAD is gradually incorporating more advanced data, model, and technology systems to enhance the efficiency and accuracy of global commodity estimates. Since 1974, NASA and the USDA have collaborated intermittently on research in remote sensing of agriculture (Hammond, 1975 and Macdonald and Hall, 1980). More recently NASA and academic research partners have been collaborating with the USDA through a series of projects that have worked to assimilate NASA products to improve IPAD’s decision support system (DSS). Operational use of new products (data, information, models) within an organization is not purely a technical issue. A decision support tool operates within a broader decision making system that is based on data and technology but is driven by the experience, preferences and perceptions of individuals and groups inside the organization who operate it. It is within this conceptual structure that any integration effort must be benchmarked to assess adverse effects, probability of success, and operational hurdles that must be overcome – technical or organizational. In this study, systematic benchmarking techniques (Lucertini et al., 1995 and Spendolini, 1992) were used to evaluate the effectiveness of risk-reducing mitigations like the assimilation and utilization of NASA Earth Observation System (EOS) (Justice et al., 1998) data. The Defect Detection and Prevention (DDP) risk management software tool developed by NASA’s Jet Propulsion Laboratory (JPL) (Cornford, 1998, Feather, 2001, Feather et al., 2000 and Gindorf and Cornford, 1999) was employed to quantify the effectiveness of the enhancements, using risk balance and attainment of objectives as performance indicators. The focus of this research was the application of a novel benchmarking approach to assess the impact of enhancements on a complex operational decision support system used by IPAD. The benchmarking process measures the change – or delta – between pre and post-enhancement states of the DSS by using metrics that quantify changes in the DSS after infusion of NASA science, data and technology. The outcome of this benchmarking is intended to quantify how investments from USDA and NASA have enhanced the performance of IPAD’s DSS. It is increasingly important for NASA’s Earth Sciences Division to integrate NASA solutions into DSSs (McCuistion & Birk, 2005). Although other agencies have also contributed to the enhancements of IPAD’s DSS, benchmarked in this study, NASA products are the focus to illustrate how environmental remote sensing science and data assist IPAD to attain its mission objectives and requirements.

FAS/IPAD is an operational division requiring continuous and consistent data products. One thread of this dynamic data stream includes redundant Earth observation data sets from AVHRR, SPOT-VEGETATION, MODIS, and future satellites like VIIRS. Weather now-casting based on GOES and Meteosat and climate forecasting models are also pertinent to IPAD’s mission. The use of decision support systems and tools often results in more informed decision making and better communication among stakeholders (Sivakami & Karthikeyan, 2009), managers and decision makers. However, to efficiently develop and adapt decision support tools assimilating satellite data (e.g. MODIS, ASTER, AMSR) and other products, partnerships between stakeholders and product developers are essential to communicate user and system requirements (Pyke et al., 2007) (Fig 2). The collaborative benchmarking activities provided not only feedback about the benefits of DSS enhancement to USDA/FAS and NASA, but facilitated communication among DSS users, developers, and USDA management that helped to suggest future avenues for system development as well as improved intra- and interagency collaboration. It is important to note that production estimates for each country, commodity and season are derived differently based on the usability and availability of information sources for each country and the unique requirements presented to its analyst. The benefits demonstrated in this benchmarking exercise of the assimilation of NASA data in IPAD’s evolving DSS are: improved quality of crop assessment and production estimates and decisions, cost reduction and timesavings. For example, benchmarking through the evaluation of performance metrics showed the effectiveness of the assimilation of MODIS VI (Vegetation Index) products into IPAD’s DSS. The MODIS vegetation products (as well as some ASTER data) are one of multiple lines of evidence that are used in a convergence of evidence methodology to estimate agricultural production at a global scale. The MODIS VI product is shared online through Cropexplorer (http://www.pecad.fas.usda.gov/cropexplorer/). Results from validation and verification efforts can provide analysts a measure of confidence in the products. The development of data fields that provide the end-user community with better quality information associated with satellite data products will nurture enhanced confidence and could move users from visual examination to more sophisticated uses and better decision making (Bahill & Gissing, 1998). The degree of structure and problem complexity for the IPAD DSS was favorably altered, changing from an unstructured, novel problem to a semi-structured, modeled problem by the introduction of products derived from NASA observations and research. Introducing such products for use in a DSS serving semi- or un-structured, complex problems can improve the quality of information on which human judgment is based by providing not only new and different information for the decision maker but also alternative solutions with their potential impacts. Improving the quality of information available from selected NASA knowledge, capacity, and systems for input to the IPAD DSS then has both strengthened the structure of the IPAD DSS problem by making it more amenable to systematic (i.e. algorithmic) solutions, and improved the information and evidence available on which to base decisions. This has a number of important ramifications, offering such prospects as: (a) Enhanced consistency of results; (b) Increased accuracy; (c) Increased throughput speed; and (d) Increased socio-economic benefit. The novel application of the DDP tool provided a means of quantifying information about critical aspects of the OGA/IPAD decision support system. For example, it became immediately obvious that the Objectives, Risks and Mitigations needed to be succinctly descriptive, define the complete but not overlapping set of affective elements, and must be to the point. Critical decisions involved determining the weight or importance of each of the Objectives identified for meeting the institutional mission, the impact of specific Risk elements on the Objectives, and the utility of various Mitigations for countering the Risks and attaining the Objectives. The DDP proved to be a robust tool for measuring and portraying changes in a complex DSS. Increasing the quality or variety of suitable Mitigations leads to greater Objectives attainment, as does simply increasing the number of Mitigations (but not for the same reasons). Objectives attainment shows a generally asymptotic trend approaching 100% OA with additional remote sensing Mitigations. Despite this behavior, the DDP data permit teasing apart the relative value of particular Mitigations and combinations of them. The DDP software provides a powerful tool to consistently and quantitatively manage and analyze risk and benchmark IPAD’s DSS. The DDP tool elicited many constructive discussions and helped direct the benchmarking thought process in a risk management context. Since NASA enhancements to IPAD’s DSS are expected to be at various stages of implementation (e.g. between State 2 and State 3), the results presented here can be complemented with annual repeat surveys or when most State 3 enhancements are implemented. The same benchmarking approach could be used to measure the difference in performance of the DSS between State 2 and State 3 and subsequent States, and can be based on performance indicators like risk balance and objective attainment using the current IPAD-DDP application as a baseline. This benchmarking of IPAD’s DSS also provides the International Earth Observing System (IEOS) with a better understanding of the observational needs of the national agricultural efficiency applications area. This work demonstrates how requirements must be solicited not only from the USDA but also with respect to global priorities, research agenda and needs, and around the alignment of international collaboration across many agencies, institutions and local user communities. Although new technology such as LIDAR currently lacks global coverage, it could provide many insights in the growth patterns and structure of crops and vegetation. RADAR is another promising technology for retrieving soil moisture and can be used to monitor crops under cloud cover. Strategic IEOS requirements could be derived from a comprehensive examination of IPAD’s integrated data, modeling, tools and user community requirements. This benchmarking approach could also provide insights to the US Office of Management and Budget (OMB) about how investments are used and aid decision making that result in improved and independent global commodity production intelligence that equally benefits US and world societies. This research resulted in a model for benchmarking Decision Support Systems within a larger framework of system solutions. It has implications for national and international agricultural policies, for agricultural production and efficiency management, for societal benefits and impacts. This benchmarking model (1) promotes intra- and inter-agency collaboration and communication, (2) accommodates multiple model results and data sources such as Earth satellite and field observations and economic data, (3) integrates scientific, visualization, system and operational requirements, (4) facilitates transfer of current and historical data and knowledge among research, management, and decision making organizations and agencies, (5) provides a risk management profile and levels of objective attainment that are associated with enhancements and changes to decision support systems. This DSS benchmarking approach is flexible; it can be applied to other decision support systems such as the Invasive Species Forecasting System (ISFS), the National Integrated Drought Information System (NIDIS), land use planning (Witlox, 2005), or other ecological or hazard forecasting systems and applications that inform management decisions.