To update the last historical year, follow the instructions below:

  1. Modify ENERGY 2100’s last historical year

The variable, XHisTime, is used throughout the model to indicate the last historical year of data.   The value of XHisTime is assigned inside Engine, in the E2020_Constants.jl file.  


To modify ENERGY 2100’s last historical year, follow the steps below.


    1. Open E2020_Constants.jl in \Engine and search for xHisTime to locate the line that assigns a year to xHisTime, for example “xHisTime = 2025”.


    1. Modify the value of xHisTime to be the new last historical year, for example “xHisTime = 2026”.  Save the file.


    1. Execute E2020_Constants.jl to verify the file compiles and creates the new last historical year. 


  1. Update raw input data with additional year of data

Most, if not all, of the Canadian input data are housed in one of three Access databases – vData.accdb, vData_CAC.accdb, or vData_ElectricUnits_CN.accdb.


Follow the steps below to update raw input data in the model.


  1. Replace the three Canadian input Access databases with new databases containing revised data in \VBInput\InputData:
      1. vData.accdb
      2. vData_CAC.accdb
      3. vData_ElectricUnits_CN.accdb


  1. Execute VBInput to read the Access input data into the Promula database: 
  1. Double-click RunVBInput.bat directly under the \VBInput subdirectory.  
  2. If an error occurs running VBInput, there is an issue with the input data.  If you can identify what the issue is, revise the Access database, then re-execute VBInput.  If you cannot identify the source of the error, contact SSI.


  1. Check data files stored in \Input\Scripts to see if any data needs to be updated.


  1. Check for syntax errors in the modified text files by compiling each individual file in Julia. Needs instruction.


  1. Modify year selections in the \Input\Scripts text files that assign variables read into Promula via VBInput (‘v’ variables) to model variables (typically ‘X’ variables).

All data read in through the VBInput program are assigned to model variables named with a ‘v’ prefix, such as vENPN or vDmd. Julia files in \Input\Scripts transfer those ‘v’ variables to model variables (usually named using an ‘x’ prefix).  For example, vENPN is written to xENPN in the WholesalePrices_VB.jl file. These data transfer files often contain year selections that may need to be updated given a different set of years of input data. For example, if vENPN data were only input through 2045, the file WholesalePrices_VB.jl may extrapolate values from 2046 to 2050. If the last year of the input data changed from 2045 to 2050, then the year selections and extrapolations in the script file would need to change.


Following the execution of VBInput, review the files in \Input\Scripts listed below to determine if any year selections need to be revised. Below is a list of the files that transfer the 'v' variables to model variables.

  • DegreeDay_VB.jl
  • DeliveredPrices_VB_US.jl
  • DeliveredPricesFuelSet_VB
  • DeviceInputs_VB.jl
  • DeviceSaturation_VB_Com.jl
  • DeviceSaturation_VB_Res.jl
  • DeviceSaturation_VB_Ind.jl
  • DeviceSaturation_VB_Trans.jl
  • EconomicDrivers_VB.jl
  • Elec_SalesPurch_VB.jl
  • Elec_UnRetire_VB.jl
  • Electric_PeaksMinimumsOutputs_VB.jl
  • ElectricPlantCharacteristics_VB.jl
  • Electric_Transmission_VB.jl
  • ElectricGeneration_VB.jl
  • ElectricUtilityFuelUse_VB.jl
  • Elec_SalesPurch_VB.jl 
  • Energy_Flows_VB.jl
  • EnergyDemand_VB_Com.jl
  • EnergyDemand_VB_Res.jl
  • EnergyDemand_VB_Ind.jl
  • EnergyDemand_VB_Trans.jl
  • FlowFraction_VB.jl
  • GasOilSupply_VB.jl
  • GHG_VB_Res.jl     
  • GHG_VB_Com.jl     
  • GHG_VB_Ind.jl     
  • GHG_VB_Trans.jl     
  • GHG_VB_ElecUtility.jl
  • GHG_VB_RCI.jl     
  • CAC_VB_RCI.jl          
  • CAC_VB_Trans.jl  
  • CAC_VB_ElecUtility.jl
  • CAC_VB_ReductionCurves.jl
  • ImportEmissions_VB.jl
  • LNGProduction_VB.jl
  • NGTransmission_VB.jl
  • OG_UnitDefinitions_VB.jl
  • OGExchangeAndInflation.jl
  • RefiningDefinitions_VB.jl
  • RPP_ProductionImportsExports_VB.jl
  • Steam_VB.jl
  • Transportation_VB.jl
  • VehicleStock_VB.jl
  • Waste_VB.jl
  • WholesalePrices_VB.jl


  1. Modify years to be calibrated given new year of data

All required historical data must be updated through the last historical year or the model may not calibrate properly.  However, the last historical calibration year can vary by sector if needed.  

Given a new year of historical data, determine and set the calibration years using the following steps.

  1. If there are any sectors that have zero demands starting in 1985 then modify the InitialDemandYear[EC,Area] variable in Calibration\AdjustInitialYear.jl. InitialDemandYear[EC,Area] indicates, for a given sector, which year to use as the first year to be used for initializing calibration equations and curves.
  2. Modify CalibTime[EC,Area] in Calibration\CalibSettings.jl.  CalibTime[EC,Area] indicates the last year to be calibrated.
  3. Modify electric load calibration years in Calibration\CalibSettings.jl using the variable CalibLTime[Area].


Revise the final year of execution to be the new last historical year.


  1. Modify the electric price calibration in RunModel.jl. 

Electric prices are calibrated in the second execution statement in RunModel.jl in the 2020Model subdirectory.  The calibration for electric prices is different in that we calibrate prices five years out into the future.  


rem  =========================

rem  Calibrate Electric Prices

rem  =========================

rem

     Call PrmFile ElecPriceSwitches.jl   

     Call ExecuteModel 1986 2015


Given a new year of historical data, modify the electric price calibration to be equal to five years past the last historical year.


  1. Modify the start year of the model execution This section needs redone

Even though there is a new last historical year, the year to start the execution of ENERGY 2020 forecast often does not change. 

    1. Change the first year of execution for the Reference case (Ref26) run by changing the start year in the calls to RunScenario and RunIntegratedPolicy statements in 2020Model\RunRef26.bat.


The non-integrated runs are executed using a call to RunScenario:

     Call RunScenario 2020 2050 Ref13 Base Base Ref13 Short


The integrated runs in Ref13 are executed using a call to RunIntegratedPolicy:

     Call RunIntegratedPolicy 2020 2050 Ref13_TIM Base_TIM Base_TIM Ref13

 F1201_EC Base


    1. Search through other batch files used to execute scenarios and change the first year to any calls to RunScenario or RunIntegratedPolicy.


  1. Finding and troubleshooting issues with historical data

Adding a new historical year requires changing a significant number of files and incorporating a large amount of new data. Occasionally, the scale of these changes results in missing data or an unintended bug being introduced when changing model files. Finding these issues can be time-consuming.  Follow the steps below when looking for and trouble-shooting issues with the historical data.

    1. Run the model incrementally when adding new data
  1. If new data is added to the vData Access databases or Superset then execute a RunAll and check for any issues using the \Process database
  2. If data is incorporated correctly then \Process can be the new starting point for new runs.  In other words, when making subsequent model runs, you can execute a RunModel.bat from \2020Model rather than a RunAll.cmd.


    1. Use the model to check for missing historical data
  1. To check if all required data is being read in for the new historical year, execute the model through the calibration (dba.zip is created in \Calib).
  2. If the model stops with an error:
        • Note the error message on screen (ex: “Could not divide by zero”). Executing non-compiled Promula will often generate an error message if the compiled outputs are not helpful.
        • Examine .log files in 2020Model folder to determine which sector is generating the error. Which file was most recently created?
        • Once sector is identified, check historical data for missing data. Are some variables missing values in the newest year?
  1. If issue persists then contact SSI for assistance


    1. Check the calibration of historical data:

Calibration is essentially the simulation of the historical period by the model to set model parameters by matching the input data. Even if historical data is present, outliers or inconsistent data can result in the model not calibrating correctly.


To check if the model calibrated, check the model’s values of historical demand, gross output, electric prices compared to the exogenous inputs of historical demand, gross output, and electric prices.  In terms of variables, this means check that Dmd matches XDmd, GO matches XGO, and PE matches XPE historically.  You can do this through the Promula interface (2020.bat) or using output files created to help check the calibration.


To check if the model calibrated, check results from the calibration output file:  CalibrationCheck.dta.


    1. Identify the data issue causing the calibration issue

If the model did not calibrate, then it is due to data inconsistencies, zeros, or anomalies.  The following is a list of common data issues.  Please contact SSI if your model is not calibrating to help identify the data source of the problem.

      • Disagreements between the input energy data and macroeconomic data in TIM. For instance, calibration issues can arise if energy usage in a sector declines but the economic driver significantly increases.
      • Disagreements between related energy data, such as if historical electric sales are significantly than electric generation values in a given area.
      • Errors in data formatting. If a decimal point in the input data is accidentally moved one spot in the new historical year then the model can have issues accounting for the large spike or drop when all other factors are held constant.
      • Adding new sectors to the historical period without any previous initialization values.


  1. Review and/or remove “Adjust” files in 2020Model

Adjustments to input data or to the forecast are typically made in text files stored in the 2020Model subdirectory and are named with the prefix “Adjust”.  These adjustment files do a variety of tasks, including fixing issues with historical data and adjusting the forecast assumptions to reflect expert thinking.  


Each time there is an update to the model with a new set of data, the adjustment files need to be reviewed to determine whether the adjustments are still relevant.  You will likely need to make revisions to the years, revisions to the assumptions, or delete the file in its entirety.


In order to determine whether an adjustment file is needed when updating the model, follow the steps below:

    1. Review Adjust files to check for Select Year statements. If the adjustment begins or ends at the last historical year then change to new value
    2. Run the model to develop a new Base scenario for comparison purposes.
    3. Remove all adjustment files from RunModel.jl and rerun the Base scenario.
    4. Compare results of the model before and after the adjustment files have been removed.
    5. If it is determined that some of the adjustment files may still be needed, then review each file for its relevance, then put the adjustments back in one at a time and rerun the Base scenario after each one and do a before and comparison before moving on to the next adjustment file.


  1. Review policy files 

Each scenario is constructed using policy files, which are created to simulate a specific event, planned policy, or general assumption about the forecast. Policy files are designed with the best possible knowledge of future events at the time they are created. Over time, the expected impacts of policies can change and diverge from the intent of the file when it was created. 


Therefore, a review of policy files and scenarios should be conducted every time the historical data is updated for the following:

    1. Does the policy start in a specific year? Is this year now in the historical period?  Should we expect to see the impact of the policy in the historical data?
    2. How has the new data altered the impact of the policy?
    3. Is the policy still relevant to the forecast scenario? How have expectations about the scenario changed over the past year?


  1. Review Base and Reference Case forecasts

Reviewing the forecast is another key check for ensuring that the new historical data is being read into the model properly. The last historical year is a central component for trends into the future, so any abnormal data can have a large influence on the forecast. 

    1. Compare new Base and Ref13 case to previous versions. Which sectors have a large change in the forecast values? Can these new trends be traced to data in the new historical year?
    2. Compare results to outside forecasts and expert analysis. Are the model outputs reasonable?


  1. Review special adjustments to finalize Reference Case forecast

To finalize the Reference case, there are several additional Reference case policies that require special adjustments.  The adjustments required need to be made after the initial Reference case has been run in order to obtain initial levels used in the policies.  The following are current examples of Reference case policies that require special adjustments.  These special adjustment files change each year, so contact SSI for the most up-to-date list of adjustments and instructions on how these are created.

    • Alberta Cap and Trade
    • OnFarm forecast
    • Transportation Road Dust driver
    • HFC emissions forecast