A model can never perfectly represent a natural system. Due to paucity of data, errors in system definitions and data used as well as limited system understanding, uncertainty us inherent in all MFAs. Data for material flow analysis originate from different sources and vary in terms of availability and quality, particularly if material stocks and flows of large-scale systems, such as regions or whole economies, are investigated. Ignoring uncertainty in MFA has raised doubts about the reliability of MFA results. Approaches to uncertainty analysis aim at making uncertainties transparent and reducing them. They enable the users to make more robust assumptions and to become aware of the model’s strengths and limitations. A good uncertainty analysis addresses both systematic errors (e.g., system definition) as well as random errors (e.g. data). There are different ways to deal with uncertainty.
Please note: The following text is taken from the existing MinFuture reports "A systems approach for the monitoring of the physical economy" and "Compilation of uncertainty approaches and recommendations for reporting data uncertainty". Please click on the afore-mentioned links to access and read the full publications.
Material flow analysis (MFA), is a term used by a wide range of approaches to describe material and energy stocks and flows in systems defined in space and time. A model can never perfectly represent a real system. Because of that, model predictions are always uncertain. Besides MFA concerns gathering, harmonizing, and analysing data about physical flows and stocks from different sources with varying qualities, therefore data limitations are unavoidable. Although studies of material flow systems can provide useful information, they also depend on data and information and their absence can be a limiting factor. In addition, results of limited or unknown accuracy may have negative impacts on subsequent decisionmaking processes. Clearly, if MFA is seen as a way of compiling data to create information about material stocks and flows and to aggregate this information to create knowledge about material flow systems, the quality of its fundamental components, data, is crucial.
Methods to deal with uncertainty in MFA range from qualitative discussions to sophisticated statistical approaches. Different methods will be evaluated. The majority of methods can be classified into four groups:
- Data classification methods
- Uncertainty analysis approaches
- Sensitivity analysis approaches
- Comparisons of model structures
The nature of uncertainty in MFA
In MFA, uncertainty analysis should consider all available information about the system and data and reflect its purpose and data quality. The uncertainty of the data and the accuracy of the results are fundamental derivatives of the evaluation process. As MFA concerns gathering, harmonizing and analysing data about physical stocks and flows from various different sources with varying quality, limitations of data are unavoidable. The majority of data used in MFA are empirical quantities with uncertainty arising from different sources. The causes, sources and types of uncertainty are summarised in Table 1, Table 2 and Table 3 below.
Table 1: Causes of Uncertainty
Table 2: Sources of uncertainty
Table 3: Types of uncertainty
Ways to deal with uncertainty
Statistical analysis uses inaccurate data samples to get as decent as possible knowledge of one entity. If there is enough data available, it is possible to use statistics (median, standard deviation). In some situations, however, problems may occur with statistical methods, therefore, different approaches to treat uncertain data have been developed. Methods to deal with uncertainty in MFA range from qualitative discussions to sophisticated statistical approaches, see Table 4 below.
Uncertainty is often characterized without the use of formal procedures, which impairs statements about the reliability of the MFA results based on uncertainty analysis. Therefore, consistent and transparent procedures are imperative for uncertainty analysis in MFA. In addition, different uncertainty types need to be addressed by different concepts used to express and propagate uncertainty.
A systematic iterative procedure for handling uncertainty in MFA is presented in Figure 14. Depending on the specific application, only parts of the scheme may be completed. Some studies may not warrant full uncertainty analysis because of different emphases of the MFAs. If the MFA is mainly used to quantify material balances to improve the database, steps 1 to 3 are of major importance and step 4 resembles the uncertainty result. If the goal of the MFA does not require describing the inherent uncertainty of model outputs resulting from the data used, then the characterization of data uncertainty (step 2) is less important, but sensitivity analysis and/or scenario modelling (step 5) are important elements to be taken into consideration.
Table 4: Approaches to deal with Uncertainty
Figure 8: Schematic illustration of a systematic procedure for uncertainty analysis in MFA
The schematic framework in Figure 8 under Approaches above, should be considered when incorporating uncertainty analysis in MFA, but could also be used to justify the approach followed for quantifying uncertainty in a study. In the pyramid, uncertainty is associated with all of its components as illustrated below.
- System: A model is a simplified version of the real system, which is difficult to replicate perfectly and therefore uncertainty is inherently present. The first step for handling uncertainty in MFA is to define the elements of the system and the mathematical relationships between them in consideration of the mass balance principle.
- Data: Because data often originates from different sources, it is unavoidably of varying quality. To deal with uncertain data appropriately, functions need to be characterised. If sufficient empirical evidence is available, statistical parameter estimation techniques or goodness-of-fit tests can be applied.
- Model: Assumptions and simplifications are made that lead to uncertainty regarding the validity of the model predictions to reflect real world situations. Usually, model equations are solved numerically and therefore associated uncertainty is low. For exploratory MFA, sensitivity analysis is used to evaluate the effects of parameter variation on the model outputs. It forms the basis to identify critical model parameters, which can influence the material flows in the system and therefore the final outcomes.
- Indicators: The calibrated model, where satisfying agreement between data and model results has been achieved, is used to calculate the model outputs (material flows and associated uncertainty). Indicators and their associated uncertainty can therefore be calculated. It is also possible to interpret uncertainty estimates for the resulting flows.
- Visualisation: It is possible to visualise uncertainty in many different ways, depending on the kind of visualisation of the MFA itself (e.g. resolution), the related results (e.g. Sankey, Pie, paired bar, maps, stacked column) and the available data.
- Decision-making: Although studies of material flow systems can provide information and convey knowledge, they also depend on information and knowledge in their production process. A lack of useful information can be a limiting factor about the level of detail provided in an analysis with subsequent influence on decisions to be taken when considering such result. New challenges emerge when the qualitative discussions of political science meet the quantitative approach of physical science. There seem today to be little support for how few, subjectively estimated data with large uncertainties should be taken into account. Still, there is a need to consider and calculate results from uncertain data. It is important therefore to convince data providers to include data uncertainty in their publications. In addition, often decision-making, whether this is policy or strategy development have a broader scope and focus and may account for multiple parameters that have not been taken into consideration in the model development (e.g. social impacts, job opportunities etc). In an ideal scenario, the scope of a study should be informed about parameters of influence that are beyond the scope of MFA to account for uncertainty accordingly.
In conclusion, there are a handful of applicable approaches to consider data uncertainty in MFA. The employment of MFA software would facilitate the implementation of these approaches and reduce the additional workload. However, such software support is not yet
on the market (with the exemption of STAN3, which is limited to normally distributed values) and there is a strong necessity to fund such development. Only then, MFA could enter into an era where reporting uncertainty ranges of stocks and flows is integrated during
the study development stage. This would help to judge or gauge the reliability of MFA studies and allow comparative studies to take place.
- The systematic evaluation of uncertainty in MFA is important to understand the robustness of material flow estimates, independent of the approach used for expressing uncertainty.
- Consistent and transparent procedures for uncertainty characterization are imperative for uncertainty analysis in MFA.
- Uncertainty should be seen as a red thread throughout the modelling study starting from the very beginning.
The intention of this conference "Monitoring the physical economy requires putting raw material data in a system context" was to present, reflect on and refine key recommendations from the MinFuture project on a monitoring system for the physical economy.
The presentations held and recommendations made will soon be available via this website.
This report aims to address existing challenges by developing a methodological framework for the monitoring of the physical economy that facilitates the users in reflecting more systematically about the problems mentioned above and in developing more effective strategies for addressing them. The framework proposed is based on Material Flow Analysis (MFA), a tool widely used for tracking materials and energy in the economy.
Six material-specific workshop were organised under the umbrella of MinFuture project, aiming to identify gaps that hamper a robust mapping of raw material cycles. In each material-specific workshop, stakeholders from academia, industry, and government were brought together to discuss and comment on the current status of each material cycle.
Material flow analysis (MFA) in general, is a term used to summarize a wide range of approaches to describe material stocks and flows in systems defined in space and time.
In order to develop strategies as well as to define and reach goals concerning raw materials management, maps are needed to help navigate existing knowledge and data.