Building Aggregation Maps From Row and Column Labels
building_aggregation_maps.RmdIntroduction
Aggregation maps are helpful when aggregating products and industries in rows and columns of PSUT matrices (R, U, V, and Y). This vignette demonstrates ways to use row and column names from PSUT matrices to create an aggregation map.
To provide an example, this vignette uses data from one of the CL-PFU
Database products, namely the product named “psut” from v0.9 of the
database, which is a matsindf data frame containing PSUT
matrices for the USA, 1960–2019.
Load PSUT data
To work with the CL-PFU database as shown in this vignette, be sure you have access to several packages.
To load the data, use code like this:
pinboard <- pins::board_folder("~/Dropbox/Fellowship 1960-2015 PFU database/OutputData/PipelineReleases")
df <- pinboard |>
# Load USA data from v0.9 of the database
pins::pin_read(name = "psut_usa", version = "20230220T223535Z-35e3e")
dplyr::glimpse(df)
#> Rows: 720
#> Columns: 14
#> $ Country <chr> "USA", "USA", "USA", "USA", "USA", "USA", "USA", "USA", "U…
#> $ Method <chr> "PCM", "PCM", "PCM", "PCM", "PCM", "PCM", "PCM", "PCM", "P…
#> $ Energy.type <chr> "E", "E", "E", "E", "E", "E", "E", "E", "E", "E", "E", "E"…
#> $ Last.stage <chr> "Final", "Final", "Final", "Final", "Final", "Final", "Fin…
#> $ Year <dbl> 1960, 1961, 1962, 1963, 1964, 1965, 1966, 1967, 1968, 1969…
#> $ IEAMW <chr> "IEA", "IEA", "IEA", "IEA", "IEA", "IEA", "IEA", "IEA", "I…
#> $ Y <list> <<matrix[24 x 43]>>, <<matrix[24 x 47]>>, <<matrix[24 x 4…
#> $ S_units <list> <<matrix[36 x 1]>>, <<matrix[36 x 1]>>, <<matrix[36 x 1]>…
#> $ R <list> <<matrix[33 x 26]>>, <<matrix[27 x 24]>>, <<matrix[28 x 2…
#> $ U <list> <<matrix[26 x 15]>>, <<matrix[26 x 15]>>, <<matrix[26 x 1…
#> $ U_feed <list> <<matrix[21 x 15]>>, <<matrix[21 x 15]>>, <<matrix[21 x 1…
#> $ U_EIOU <list> <<matrix[8 x 6]>>, <<matrix[8 x 6]>>, <<matrix[8 x 6]>>, …
#> $ r_EIOU <list> <<matrix[26 x 15]>>, <<matrix[26 x 15]>>, <<matrix[26 x 1…
#> $ V <list> <<matrix[15 x 28]>>, <<matrix[15 x 28]>>, <<matrix[15 x 2…Extract row and column names
Recca::get_all_products_and_industries() will find the
product and industry names for each row of the matsindf data frame. The
results are placed in two new columns of the data frame named (by
default) “Product.names” and “Industry.names”.
df_with_prods_inds <- df |>
Recca::get_all_products_and_industries()
dplyr::glimpse(df_with_prods_inds)
#> Rows: 720
#> Columns: 16
#> $ Country <chr> "USA", "USA", "USA", "USA", "USA", "USA", "USA", "USA",…
#> $ Method <chr> "PCM", "PCM", "PCM", "PCM", "PCM", "PCM", "PCM", "PCM",…
#> $ Energy.type <chr> "E", "E", "E", "E", "E", "E", "E", "E", "E", "E", "E", …
#> $ Last.stage <chr> "Final", "Final", "Final", "Final", "Final", "Final", "…
#> $ Year <dbl> 1960, 1961, 1962, 1963, 1964, 1965, 1966, 1967, 1968, 1…
#> $ IEAMW <chr> "IEA", "IEA", "IEA", "IEA", "IEA", "IEA", "IEA", "IEA",…
#> $ Y <list> <<matrix[24 x 43]>>, <<matrix[24 x 47]>>, <<matrix[24 …
#> $ S_units <list> <<matrix[36 x 1]>>, <<matrix[36 x 1]>>, <<matrix[36 x …
#> $ R <list> <<matrix[33 x 26]>>, <<matrix[27 x 24]>>, <<matrix[28 …
#> $ U <list> <<matrix[26 x 15]>>, <<matrix[26 x 15]>>, <<matrix[26 …
#> $ U_feed <list> <<matrix[21 x 15]>>, <<matrix[21 x 15]>>, <<matrix[21 …
#> $ U_EIOU <list> <<matrix[8 x 6]>>, <<matrix[8 x 6]>>, <<matrix[8 x 6]>…
#> $ r_EIOU <list> <<matrix[26 x 15]>>, <<matrix[26 x 15]>>, <<matrix[26 …
#> $ V <list> <<matrix[15 x 28]>>, <<matrix[15 x 28]>>, <<matrix[15 …
#> $ Product.names <list> <"Bitumen", "Brown coal (if no detail) [from Resources…
#> $ Industry.names <list> <"Imports [of Bitumen]", "Imports [of Coke oven coke]"…Product and industry names can be quite different from row to row in the data frame, because there are many new energy carriers (products) when the Last.stage is “Useful” compared to “Final”.
# Check a row with "Final" energy as the Last.stage.
df_with_prods_inds |>
dplyr::filter(Energy.type == "E",
Last.stage == "Final",
Year == 1960,
IEAMW == "IEA") |>
magrittr::extract2("Product.names")
#> [[1]]
#> [1] "Bitumen"
#> [2] "Brown coal (if no detail) [from Resources]"
#> [3] "Coke oven coke"
#> [4] "Crude/NGL/feedstocks (if no detail)"
#> [5] "Crude/NGL/feedstocks (if no detail) [from Resources]"
#> [6] "Electricity"
#> [7] "Fuel oil"
#> [8] "Gas/diesel oil excl. biofuels"
#> [9] "Geothermal [from Resources]"
#> [10] "Hard coal (if no detail)"
#> [11] "Hard coal (if no detail) [from Resources]"
#> [12] "Hydro [from Resources]"
#> [13] "Kerosene type jet fuel excl. biofuels"
#> [14] "Liquefied petroleum gases (LPG)"
#> [15] "Lubricants"
#> [16] "Motor gasoline excl. biofuels"
#> [17] "Natural gas"
#> [18] "Natural gas [from Resources]"
#> [19] "Non-specified primary biofuels and waste"
#> [20] "Non-specified primary biofuels and waste [from Resources]"
#> [21] "Nuclear [from Resources]"
#> [22] "Other kerosene"
#> [23] "Other oil products"
#> [24] "Paraffin waxes"
#> [25] "Patent fuel"
#> [26] "Petroleum coke"
#> [27] "Blast furnace gas"
#> [28] "Brown coal (if no detail)"
#> [29] "Coke oven gas"
#> [30] "Gas coke"
#> [31] "Gas works gas"
#> [32] "Geothermal"
#> [33] "Hydro"
#> [34] "Nuclear"
#> [35] "Refinery gas"
#> [36] "Aviation gasoline"
# Check a row with "Useful" energy as the Last.stage.
df_with_prods_inds |>
dplyr::filter(Energy.type == "E",
Last.stage == "Useful",
Year == 1960,
IEAMW == "IEA") |>
magrittr::extract2("Product.names")
#> [[1]]
#> [1] "Bitumen"
#> [2] "Brown coal (if no detail) [from Resources]"
#> [3] "Coke oven coke"
#> [4] "Crude/NGL/feedstocks (if no detail)"
#> [5] "Crude/NGL/feedstocks (if no detail) [from Resources]"
#> [6] "Electricity"
#> [7] "Fuel oil"
#> [8] "Gas/diesel oil excl. biofuels"
#> [9] "Geothermal [from Resources]"
#> [10] "Hard coal (if no detail)"
#> [11] "Hard coal (if no detail) [from Resources]"
#> [12] "Hydro [from Resources]"
#> [13] "Kerosene type jet fuel excl. biofuels"
#> [14] "Liquefied petroleum gases (LPG)"
#> [15] "Lubricants"
#> [16] "Motor gasoline excl. biofuels"
#> [17] "Natural gas"
#> [18] "Natural gas [from Resources]"
#> [19] "Non-specified primary biofuels and waste"
#> [20] "Non-specified primary biofuels and waste [from Resources]"
#> [21] "Nuclear [from Resources]"
#> [22] "Other kerosene"
#> [23] "Other oil products"
#> [24] "Paraffin waxes"
#> [25] "Patent fuel"
#> [26] "Petroleum coke"
#> [27] "Aviation gasoline"
#> [28] "Blast furnace gas"
#> [29] "Brown coal (if no detail)"
#> [30] "Coke oven gas"
#> [31] "Gas coke"
#> [32] "Gas works gas"
#> [33] "Geothermal"
#> [34] "HPA [from Air compressors]"
#> [35] "HPL [from Electric pumps]"
#> [36] "HPL [from Gas pump jacks]"
#> [37] "HTH.1600.C [from Blast oxygen furnaces]"
#> [38] "HTH.600.C [from Industrial electric process heaters]"
#> [39] "HTH.850.C [from Steam cracking furnaces]"
#> [40] "HTH.960.C [from Electro-chemical processors]"
#> [41] "Hydro"
#> [42] "IP [from Computers]"
#> [43] "KE [from Industrial fans]"
#> [44] "L [from Industrial electric lamps]"
#> [45] "LTC.-10.C [from Industrial refrigerators]"
#> [46] "LTC.20.C [from HVAC]"
#> [47] "LTH.20.C [from Gas space heaters]"
#> [48] "MacW [from Machine tools]"
#> [49] "MacW [from Material handlers]"
#> [50] "MTH.100.C [from Gas water heaters]"
#> [51] "Nuclear"
#> [52] "Refinery gas"
#> [53] "RoP [from Electric cars]"
#> [54] "AirP [from Domestic aircraft]"
#> [55] "HTH.1300.C [from Small scale biomass boilers]"
#> [56] "HTH.1600.C [from Electric arc furnaces]"
#> [57] "L [from Commercial indoor electric lamps]"
#> [58] "L [from Commercial outdoor electric lamps]"
#> [59] "L [from Domestic electric lamps]"
#> [60] "LTC.-10.C [from Refrigerators]"
#> [61] "LTC.20.C [from Room air conditioners]"
#> [62] "LTH.20.C [from Coal stoves]"
#> [63] "LTH.20.C [from Electric space heaters]"
#> [64] "LTH.20.C [from LPG stoves]"
#> [65] "LTH.20.C [from Oil space heaters]"
#> [66] "LTH.50.C [from Coal stoves]"
#> [67] "LTH.50.C [from LPG stoves]"
#> [68] "MacW [from Tractors]"
#> [69] "MaP [from Domestic boats]"
#> [70] "MD [from Domestic appliances]"
#> [71] "MD [from Static diesel engines]"
#> [72] "MD [from Static gasoline engines]"
#> [73] "MTH.100.C [from Coal stoves]"
#> [74] "MTH.100.C [from Electric water heaters]"
#> [75] "MTH.100.C [from Kerosene stoves]"
#> [76] "MTH.100.C [from LPG stoves]"
#> [77] "MTH.100.C [from Oil water heaters]"
#> [78] "MTH.100.C [from Wood stoves]"
#> [79] "MTH.200.C [from Coal stoves]"
#> [80] "MTH.200.C [from Commercial electric ovens]"
#> [81] "MTH.200.C [from Commercial gas ovens]"
#> [82] "MTH.200.C [from Domestic electric ovens]"
#> [83] "MTH.200.C [from Domestic gas ovens]"
#> [84] "NEU [from Non-energy consumption]"
#> [85] "RaP [from Diesel trains]"
#> [86] "RaP [from Electric trains]"
#> [87] "RaP [from Steam trains]"
#> [88] "RoP [from Diesel HDVs]"
#> [89] "RoP [from Gasoline LDVs]"
#> [90] "RoP [from Gasoline motorcycles]"
#> [91] "RoP [from Natural gas vehicles]"Similarly, industry names may be quite different when Last.stage is “Final” or “Useful”.
# Check a row with "Final" energy as the Last.stage.
df_with_prods_inds |>
dplyr::filter(Energy.type == "E",
Last.stage == "Final",
Year == 1960,
IEAMW == "IEA") |>
magrittr::extract2("Industry.names") |>
unlist()
#> [1] "Imports [of Bitumen]"
#> [2] "Imports [of Coke oven coke]"
#> [3] "Imports [of Crude/NGL/feedstocks]"
#> [4] "Imports [of Electricity]"
#> [5] "Imports [of Fuel oil]"
#> [6] "Imports [of Gas/diesel oil excl]"
#> [7] "Imports [of Hard coal]"
#> [8] "Imports [of Kerosene type jet fuel excl]"
#> [9] "Imports [of Liquefied petroleum gases]"
#> [10] "Imports [of Lubricants]"
#> [11] "Imports [of Motor gasoline excl]"
#> [12] "Imports [of Natural gas]"
#> [13] "Imports [of Other kerosene]"
#> [14] "Imports [of Other oil products]"
#> [15] "Imports [of Paraffin waxes]"
#> [16] "Imports [of Patent fuel]"
#> [17] "Resources [of Brown coal (if no detail)]"
#> [18] "Resources [of Crude/NGL/feedstocks (if no detail)]"
#> [19] "Resources [of Geothermal]"
#> [20] "Resources [of Hard coal (if no detail)]"
#> [21] "Resources [of Hydro]"
#> [22] "Resources [of Natural gas]"
#> [23] "Resources [of Non-specified primary biofuels and waste]"
#> [24] "Resources [of Nuclear]"
#> [25] "Statistical differences"
#> [26] "Stock changes [of Bitumen]"
#> [27] "Stock changes [of Crude/NGL/feedstocks]"
#> [28] "Stock changes [of Fuel oil]"
#> [29] "Stock changes [of Gas/diesel oil excl]"
#> [30] "Stock changes [of Hard coal]"
#> [31] "Stock changes [of Kerosene type jet fuel excl]"
#> [32] "Stock changes [of Other oil products]"
#> [33] "Stock changes [of Petroleum coke]"
#> [34] "Blast furnaces"
#> [35] "Coal mines"
#> [36] "Coke ovens"
#> [37] "Gas works"
#> [38] "Main activity producer electricity plants"
#> [39] "Manufacture [of Geothermal]"
#> [40] "Manufacture [of Hydro]"
#> [41] "Manufacture [of Non-specified primary biofuels and waste]"
#> [42] "Manufacture [of Nuclear]"
#> [43] "Natural gas extraction"
#> [44] "Nuclear industry"
#> [45] "Oil extraction"
#> [46] "Oil refineries"
#> [47] "Patent fuel plants"
#> [48] "Transfers"
#> [49] "Agriculture/forestry"
#> [50] "Chemical and petrochemical"
#> [51] "Commercial and public services"
#> [52] "Domestic aviation"
#> [53] "Domestic navigation"
#> [54] "Exports [of Bitumen]"
#> [55] "Exports [of Coke oven coke]"
#> [56] "Exports [of Crude/NGL/feedstocks]"
#> [57] "Exports [of Electricity]"
#> [58] "Exports [of Fuel oil]"
#> [59] "Exports [of Gas/diesel oil excl]"
#> [60] "Exports [of Hard coal]"
#> [61] "Exports [of Kerosene type jet fuel excl]"
#> [62] "Exports [of Liquefied petroleum gases]"
#> [63] "Exports [of Lubricants]"
#> [64] "Exports [of Motor gasoline excl]"
#> [65] "Exports [of Natural gas]"
#> [66] "Exports [of Other kerosene]"
#> [67] "Exports [of Other oil products]"
#> [68] "Exports [of Paraffin waxes]"
#> [69] "Exports [of Patent fuel]"
#> [70] "Exports [of Petroleum coke]"
#> [71] "Exports to World aviation bunkers [of Kerosene type jet fuel excl]"
#> [72] "Exports to World marine bunkers [of Fuel oil]"
#> [73] "Exports to World marine bunkers [of Gas/diesel oil excl]"
#> [74] "Exports to World marine bunkers [of Hard coal]"
#> [75] "Final consumption not elsewhere specified"
#> [76] "Industry not elsewhere specified"
#> [77] "Iron and steel"
#> [78] "Losses"
#> [79] "Non-energy use in transport"
#> [80] "Non-energy use industry/transformation/energy"
#> [81] "Rail"
#> [82] "Residential"
#> [83] "Road"
#> [84] "Stock changes [of Coke oven coke]"
#> [85] "Stock changes [of Liquefied petroleum gases]"
#> [86] "Stock changes [of Lubricants]"
#> [87] "Stock changes [of Motor gasoline excl]"
#> [88] "Stock changes [of Natural gas]"
#> [89] "Stock changes [of Other kerosene]"
#> [90] "Stock changes [of Paraffin waxes]"
# Check a row with "Useful" energy as the Last.stage.
df_with_prods_inds |>
dplyr::filter(Energy.type == "E",
Last.stage == "Useful",
Year == 1960,
IEAMW == "IEA") |>
magrittr::extract2("Industry.names") |>
unlist()
#> [1] "Imports [of Bitumen]"
#> [2] "Imports [of Coke oven coke]"
#> [3] "Imports [of Crude/NGL/feedstocks]"
#> [4] "Imports [of Electricity]"
#> [5] "Imports [of Fuel oil]"
#> [6] "Imports [of Gas/diesel oil excl]"
#> [7] "Imports [of Hard coal]"
#> [8] "Imports [of Kerosene type jet fuel excl]"
#> [9] "Imports [of Liquefied petroleum gases]"
#> [10] "Imports [of Lubricants]"
#> [11] "Imports [of Motor gasoline excl]"
#> [12] "Imports [of Natural gas]"
#> [13] "Imports [of Other kerosene]"
#> [14] "Imports [of Other oil products]"
#> [15] "Imports [of Paraffin waxes]"
#> [16] "Imports [of Patent fuel]"
#> [17] "Resources [of Brown coal (if no detail)]"
#> [18] "Resources [of Crude/NGL/feedstocks (if no detail)]"
#> [19] "Resources [of Geothermal]"
#> [20] "Resources [of Hard coal (if no detail)]"
#> [21] "Resources [of Hydro]"
#> [22] "Resources [of Natural gas]"
#> [23] "Resources [of Non-specified primary biofuels and waste]"
#> [24] "Resources [of Nuclear]"
#> [25] "Statistical differences"
#> [26] "Stock changes [of Bitumen]"
#> [27] "Stock changes [of Crude/NGL/feedstocks]"
#> [28] "Stock changes [of Fuel oil]"
#> [29] "Stock changes [of Gas/diesel oil excl]"
#> [30] "Stock changes [of Hard coal]"
#> [31] "Stock changes [of Kerosene type jet fuel excl]"
#> [32] "Stock changes [of Other oil products]"
#> [33] "Stock changes [of Petroleum coke]"
#> [34] "Air compressors -> HPA"
#> [35] "Blast furnaces"
#> [36] "Blast oxygen furnaces -> HTH.1600.C"
#> [37] "Coal mines"
#> [38] "Coal stoves -> LTH.20.C"
#> [39] "Coal stoves -> LTH.50.C"
#> [40] "Coal stoves -> MTH.100.C"
#> [41] "Coal stoves -> MTH.200.C"
#> [42] "Coke ovens"
#> [43] "Commercial electric ovens -> MTH.200.C"
#> [44] "Commercial gas ovens -> MTH.200.C"
#> [45] "Commercial indoor electric lamps -> L"
#> [46] "Commercial outdoor electric lamps -> L"
#> [47] "Computers -> IP"
#> [48] "Diesel HDVs -> RoP"
#> [49] "Diesel trains -> RaP"
#> [50] "Domestic aircraft -> AirP"
#> [51] "Domestic appliances -> MD"
#> [52] "Domestic boats -> MaP"
#> [53] "Domestic electric lamps -> L"
#> [54] "Domestic electric ovens -> MTH.200.C"
#> [55] "Domestic gas ovens -> MTH.200.C"
#> [56] "Electric arc furnaces -> HTH.1600.C"
#> [57] "Electric cars -> RoP"
#> [58] "Electric pumps -> HPL"
#> [59] "Electric space heaters -> LTH.20.C"
#> [60] "Electric trains -> RaP"
#> [61] "Electric water heaters -> MTH.100.C"
#> [62] "Electro-chemical processors -> HTH.960.C"
#> [63] "Gas pump jacks -> HPL"
#> [64] "Gas space heaters -> LTH.20.C"
#> [65] "Gas water heaters -> MTH.100.C"
#> [66] "Gas works"
#> [67] "Gasoline LDVs -> RoP"
#> [68] "Gasoline motorcycles -> RoP"
#> [69] "HVAC -> LTC.20.C"
#> [70] "Industrial electric lamps -> L"
#> [71] "Industrial electric process heaters -> HTH.600.C"
#> [72] "Industrial fans -> KE"
#> [73] "Industrial refrigerators -> LTC.-10.C"
#> [74] "Kerosene stoves -> MTH.100.C"
#> [75] "LPG stoves -> LTH.20.C"
#> [76] "LPG stoves -> LTH.50.C"
#> [77] "LPG stoves -> MTH.100.C"
#> [78] "Machine tools -> MacW"
#> [79] "Main activity producer electricity plants"
#> [80] "Manufacture [of Geothermal]"
#> [81] "Manufacture [of Hydro]"
#> [82] "Manufacture [of Non-specified primary biofuels and waste]"
#> [83] "Manufacture [of Nuclear]"
#> [84] "Material handlers -> MacW"
#> [85] "Natural gas extraction"
#> [86] "Natural gas vehicles -> RoP"
#> [87] "Non-energy consumption -> NEU"
#> [88] "Nuclear industry"
#> [89] "Oil extraction"
#> [90] "Oil refineries"
#> [91] "Oil space heaters -> LTH.20.C"
#> [92] "Oil water heaters -> MTH.100.C"
#> [93] "Patent fuel plants"
#> [94] "Refrigerators -> LTC.-10.C"
#> [95] "Room air conditioners -> LTC.20.C"
#> [96] "Small scale biomass boilers -> HTH.1300.C"
#> [97] "Static diesel engines -> MD"
#> [98] "Static gasoline engines -> MD"
#> [99] "Steam cracking furnaces -> HTH.850.C"
#> [100] "Steam trains -> RaP"
#> [101] "Tractors -> MacW"
#> [102] "Transfers"
#> [103] "Wood stoves -> MTH.100.C"
#> [104] "Agriculture/forestry"
#> [105] "Chemical and petrochemical"
#> [106] "Commercial and public services"
#> [107] "Domestic aviation"
#> [108] "Domestic navigation"
#> [109] "Exports [of Bitumen]"
#> [110] "Exports [of Coke oven coke]"
#> [111] "Exports [of Crude/NGL/feedstocks]"
#> [112] "Exports [of Electricity]"
#> [113] "Exports [of Fuel oil]"
#> [114] "Exports [of Gas/diesel oil excl]"
#> [115] "Exports [of Hard coal]"
#> [116] "Exports [of Kerosene type jet fuel excl]"
#> [117] "Exports [of Liquefied petroleum gases]"
#> [118] "Exports [of Lubricants]"
#> [119] "Exports [of Motor gasoline excl]"
#> [120] "Exports [of Natural gas]"
#> [121] "Exports [of Other kerosene]"
#> [122] "Exports [of Other oil products]"
#> [123] "Exports [of Paraffin waxes]"
#> [124] "Exports [of Patent fuel]"
#> [125] "Exports [of Petroleum coke]"
#> [126] "Exports to World aviation bunkers [of Kerosene type jet fuel excl]"
#> [127] "Exports to World marine bunkers [of Fuel oil]"
#> [128] "Exports to World marine bunkers [of Gas/diesel oil excl]"
#> [129] "Exports to World marine bunkers [of Hard coal]"
#> [130] "Final consumption not elsewhere specified"
#> [131] "Industry not elsewhere specified"
#> [132] "Iron and steel"
#> [133] "Losses"
#> [134] "Non-energy use in transport"
#> [135] "Non-energy use industry/transformation/energy"
#> [136] "Rail"
#> [137] "Residential"
#> [138] "Road"
#> [139] "Stock changes [of Coke oven coke]"
#> [140] "Stock changes [of Liquefied petroleum gases]"
#> [141] "Stock changes [of Lubricants]"
#> [142] "Stock changes [of Motor gasoline excl]"
#> [143] "Stock changes [of Natural gas]"
#> [144] "Stock changes [of Other kerosene]"
#> [145] "Stock changes [of Paraffin waxes]"Get lists of unique product and industry names
To obtain lists of unique product and industry names, one can use
unique().
# Unique products in the data frame
df_with_prods_inds$Product.names |>
unlist() |>
unique()
#> [1] "Bitumen"
#> [2] "Brown coal (if no detail) [from Resources]"
#> [3] "Coke oven coke"
#> [4] "Crude/NGL/feedstocks (if no detail)"
#> [5] "Crude/NGL/feedstocks (if no detail) [from Resources]"
#> [6] "Electricity"
#> [7] "Fuel oil"
#> [8] "Gas/diesel oil excl. biofuels"
#> [9] "Geothermal [from Resources]"
#> [10] "Hard coal (if no detail)"
#> [11] "Hard coal (if no detail) [from Resources]"
#> [12] "Hydro [from Resources]"
#> [13] "Kerosene type jet fuel excl. biofuels"
#> [14] "Liquefied petroleum gases (LPG)"
#> [15] "Lubricants"
#> [16] "Motor gasoline excl. biofuels"
#> [17] "Natural gas"
#> [18] "Natural gas [from Resources]"
#> [19] "Non-specified primary biofuels and waste"
#> [20] "Non-specified primary biofuels and waste [from Resources]"
#> [21] "Nuclear [from Resources]"
#> [22] "Other kerosene"
#> [23] "Other oil products"
#> [24] "Paraffin waxes"
#> [25] "Patent fuel"
#> [26] "Petroleum coke"
#> [27] "Blast furnace gas"
#> [28] "Brown coal (if no detail)"
#> [29] "Coke oven gas"
#> [30] "Gas coke"
#> [31] "Gas works gas"
#> [32] "Geothermal"
#> [33] "Hydro"
#> [34] "Nuclear"
#> [35] "Refinery gas"
#> [36] "Aviation gasoline"
#> [37] "Naphtha"
#> [38] "Crude oil"
#> [39] "Crude oil [from Resources]"
#> [40] "Natural gas liquids [from Resources]"
#> [41] "Natural gas liquids"
#> [42] "Refinery feedstocks"
#> [43] "Coking coal"
#> [44] "Coking coal [from Resources]"
#> [45] "Gasoline type jet fuel"
#> [46] "Lignite [from Resources]"
#> [47] "Other bituminous coal"
#> [48] "Other bituminous coal [from Resources]"
#> [49] "Other hydrocarbons [from Resources]"
#> [50] "Sub-bituminous coal [from Resources]"
#> [51] "White spirit & SBP"
#> [52] "Lignite"
#> [53] "Other hydrocarbons"
#> [54] "Sub-bituminous coal"
#> [55] "Heat"
#> [56] "Solar photovoltaics [from Resources]"
#> [57] "Solar photovoltaics"
#> [58] "Solar thermal [from Resources]"
#> [59] "Wind [from Resources]"
#> [60] "Solar thermal"
#> [61] "Wind"
#> [62] "Biogases [from Resources]"
#> [63] "Ethane"
#> [64] "Industrial waste [from Resources]"
#> [65] "Municipal waste (non-renewable) [from Resources]"
#> [66] "Municipal waste (renewable) [from Resources]"
#> [67] "Primary solid biofuels"
#> [68] "Primary solid biofuels [from Resources]"
#> [69] "Biogases"
#> [70] "Industrial waste"
#> [71] "Municipal waste (non-renewable)"
#> [72] "Municipal waste (renewable)"
#> [73] "Peat [from Resources]"
#> [74] "Peat"
#> [75] "Additives/blending components"
#> [76] "Additives/blending components [from Resources]"
#> [77] "Biogasoline"
#> [78] "Biogasoline [from Resources]"
#> [79] "Biodiesels"
#> [80] "Biodiesels [from Resources]"
#> [81] "Other liquid biofuels"
#> [82] "Other liquid biofuels [from Resources]"
#> [83] "Anthracite"
#> [84] "Anthracite [from Resources]"
#> [85] "HPA [from Air compressors]"
#> [86] "HPL [from Electric pumps]"
#> [87] "HPL [from Gas pump jacks]"
#> [88] "HTH.1600.C [from Blast oxygen furnaces]"
#> [89] "HTH.600.C [from Industrial electric process heaters]"
#> [90] "HTH.850.C [from Steam cracking furnaces]"
#> [91] "HTH.960.C [from Electro-chemical processors]"
#> [92] "IP [from Computers]"
#> [93] "KE [from Industrial fans]"
#> [94] "L [from Industrial electric lamps]"
#> [95] "LTC.-10.C [from Industrial refrigerators]"
#> [96] "LTC.20.C [from HVAC]"
#> [97] "LTH.20.C [from Gas space heaters]"
#> [98] "MacW [from Machine tools]"
#> [99] "MacW [from Material handlers]"
#> [100] "MTH.100.C [from Gas water heaters]"
#> [101] "RoP [from Electric cars]"
#> [102] "AirP [from Domestic aircraft]"
#> [103] "HTH.1300.C [from Small scale biomass boilers]"
#> [104] "HTH.1600.C [from Electric arc furnaces]"
#> [105] "L [from Commercial indoor electric lamps]"
#> [106] "L [from Commercial outdoor electric lamps]"
#> [107] "L [from Domestic electric lamps]"
#> [108] "LTC.-10.C [from Refrigerators]"
#> [109] "LTC.20.C [from Room air conditioners]"
#> [110] "LTH.20.C [from Coal stoves]"
#> [111] "LTH.20.C [from Electric space heaters]"
#> [112] "LTH.20.C [from LPG stoves]"
#> [113] "LTH.20.C [from Oil space heaters]"
#> [114] "LTH.50.C [from Coal stoves]"
#> [115] "LTH.50.C [from LPG stoves]"
#> [116] "MacW [from Tractors]"
#> [117] "MaP [from Domestic boats]"
#> [118] "MD [from Domestic appliances]"
#> [119] "MD [from Static diesel engines]"
#> [120] "MD [from Static gasoline engines]"
#> [121] "MTH.100.C [from Coal stoves]"
#> [122] "MTH.100.C [from Electric water heaters]"
#> [123] "MTH.100.C [from Kerosene stoves]"
#> [124] "MTH.100.C [from LPG stoves]"
#> [125] "MTH.100.C [from Oil water heaters]"
#> [126] "MTH.100.C [from Wood stoves]"
#> [127] "MTH.200.C [from Coal stoves]"
#> [128] "MTH.200.C [from Commercial electric ovens]"
#> [129] "MTH.200.C [from Commercial gas ovens]"
#> [130] "MTH.200.C [from Domestic electric ovens]"
#> [131] "MTH.200.C [from Domestic gas ovens]"
#> [132] "NEU [from Non-energy consumption]"
#> [133] "RaP [from Diesel trains]"
#> [134] "RaP [from Electric trains]"
#> [135] "RaP [from Steam trains]"
#> [136] "RoP [from Diesel HDVs]"
#> [137] "RoP [from Gasoline LDVs]"
#> [138] "RoP [from Gasoline motorcycles]"
#> [139] "RoP [from Natural gas vehicles]"
#> [140] "MacW [from Electric cable shovels]"
#> [141] "MF [from Electric continuous miners]"
#> [142] "MF [from Ore grinding mills]"
#> [143] "HPNG [from Pipeline gas compressors]"
#> [144] "HTH.1000.C [from Clinker kilns]"
#> [145] "HTH.400.C [from Ammonia synthesizers]"
#> [146] "HTH.960.C [from Electrolytic smelters]"
#> [147] "MacW [from Paper machines]"
#> [148] "MTH.200.C [from Bayer kilns]"
#> [149] "MTH.300.C [from Heat machines]"
#> [150] "MacW [from Diesel bulldozers]"
#> [151] "MacW [from Diesel hydraulic mobiles]"
#> [152] "RoP [from Diesel haulage trucks]"
#> [153] "LTH.20.C [from Ground source heat pumps]"
#> [154] "LTH.20.C [from Wood stoves]"
#> [155] "LTH.50.C [from Wood stoves]"
#> [156] "LTH.60.C [from Solar thermal water heaters]"
#> [157] "MacW [from Electric cable cranes]"
#> [158] "MacW [from Electric hand tools]"
#> [159] "MacW [from Electric hydraulic mobiles]"
#> [160] "MF [from Diesel rock borers]"
#> [161] "HPL [from Diesel pumps]"
#> [162] "Biomass [from Resources]"
#> [163] "Biomass"
#> [164] "Feed"
#> [165] "Food"
#> [166] "AnMech [from Asses]"
#> [167] "AnMech [from Buffaloes]"
#> [168] "AnMech [from Camelids]"
#> [169] "AnMech [from Cattle]"
#> [170] "AnMech [from Horses]"
#> [171] "AnMech [from Mules]"
#> [172] "HuMech [from Human females]"
#> [173] "HuMech [from Human males]"
# Unique industries in the data frame
df_with_prods_inds$Industry.names |>
unlist() |>
unique()
#> [1] "Imports [of Bitumen]"
#> [2] "Imports [of Coke oven coke]"
#> [3] "Imports [of Crude/NGL/feedstocks]"
#> [4] "Imports [of Electricity]"
#> [5] "Imports [of Fuel oil]"
#> [6] "Imports [of Gas/diesel oil excl]"
#> [7] "Imports [of Hard coal]"
#> [8] "Imports [of Kerosene type jet fuel excl]"
#> [9] "Imports [of Liquefied petroleum gases]"
#> [10] "Imports [of Lubricants]"
#> [11] "Imports [of Motor gasoline excl]"
#> [12] "Imports [of Natural gas]"
#> [13] "Imports [of Other kerosene]"
#> [14] "Imports [of Other oil products]"
#> [15] "Imports [of Paraffin waxes]"
#> [16] "Imports [of Patent fuel]"
#> [17] "Resources [of Brown coal (if no detail)]"
#> [18] "Resources [of Crude/NGL/feedstocks (if no detail)]"
#> [19] "Resources [of Geothermal]"
#> [20] "Resources [of Hard coal (if no detail)]"
#> [21] "Resources [of Hydro]"
#> [22] "Resources [of Natural gas]"
#> [23] "Resources [of Non-specified primary biofuels and waste]"
#> [24] "Resources [of Nuclear]"
#> [25] "Statistical differences"
#> [26] "Stock changes [of Bitumen]"
#> [27] "Stock changes [of Crude/NGL/feedstocks]"
#> [28] "Stock changes [of Fuel oil]"
#> [29] "Stock changes [of Gas/diesel oil excl]"
#> [30] "Stock changes [of Hard coal]"
#> [31] "Stock changes [of Kerosene type jet fuel excl]"
#> [32] "Stock changes [of Other oil products]"
#> [33] "Stock changes [of Petroleum coke]"
#> [34] "Blast furnaces"
#> [35] "Coal mines"
#> [36] "Coke ovens"
#> [37] "Gas works"
#> [38] "Main activity producer electricity plants"
#> [39] "Manufacture [of Geothermal]"
#> [40] "Manufacture [of Hydro]"
#> [41] "Manufacture [of Non-specified primary biofuels and waste]"
#> [42] "Manufacture [of Nuclear]"
#> [43] "Natural gas extraction"
#> [44] "Nuclear industry"
#> [45] "Oil extraction"
#> [46] "Oil refineries"
#> [47] "Patent fuel plants"
#> [48] "Transfers"
#> [49] "Agriculture/forestry"
#> [50] "Chemical and petrochemical"
#> [51] "Commercial and public services"
#> [52] "Domestic aviation"
#> [53] "Domestic navigation"
#> [54] "Exports [of Bitumen]"
#> [55] "Exports [of Coke oven coke]"
#> [56] "Exports [of Crude/NGL/feedstocks]"
#> [57] "Exports [of Electricity]"
#> [58] "Exports [of Fuel oil]"
#> [59] "Exports [of Gas/diesel oil excl]"
#> [60] "Exports [of Hard coal]"
#> [61] "Exports [of Kerosene type jet fuel excl]"
#> [62] "Exports [of Liquefied petroleum gases]"
#> [63] "Exports [of Lubricants]"
#> [64] "Exports [of Motor gasoline excl]"
#> [65] "Exports [of Natural gas]"
#> [66] "Exports [of Other kerosene]"
#> [67] "Exports [of Other oil products]"
#> [68] "Exports [of Paraffin waxes]"
#> [69] "Exports [of Patent fuel]"
#> [70] "Exports [of Petroleum coke]"
#> [71] "Exports to World aviation bunkers [of Kerosene type jet fuel excl]"
#> [72] "Exports to World marine bunkers [of Fuel oil]"
#> [73] "Exports to World marine bunkers [of Gas/diesel oil excl]"
#> [74] "Exports to World marine bunkers [of Hard coal]"
#> [75] "Final consumption not elsewhere specified"
#> [76] "Industry not elsewhere specified"
#> [77] "Iron and steel"
#> [78] "Losses"
#> [79] "Non-energy use in transport"
#> [80] "Non-energy use industry/transformation/energy"
#> [81] "Rail"
#> [82] "Residential"
#> [83] "Road"
#> [84] "Stock changes [of Coke oven coke]"
#> [85] "Stock changes [of Liquefied petroleum gases]"
#> [86] "Stock changes [of Lubricants]"
#> [87] "Stock changes [of Motor gasoline excl]"
#> [88] "Stock changes [of Natural gas]"
#> [89] "Stock changes [of Other kerosene]"
#> [90] "Stock changes [of Paraffin waxes]"
#> [91] "Imports [of Naphtha]"
#> [92] "Stock changes [of Naphtha]"
#> [93] "Stock changes [of Aviation gasoline]"
#> [94] "Exports [of Aviation gasoline]"
#> [95] "Exports [of Naphtha]"
#> [96] "Exports [of Gas works gas]"
#> [97] "Pipeline transport"
#> [98] "Imports [of Crude oil]"
#> [99] "Resources [of Crude oil]"
#> [100] "Resources [of Natural gas liquids]"
#> [101] "Exports [of Crude oil]"
#> [102] "Food and tobacco"
#> [103] "Machinery"
#> [104] "Mining and quarrying"
#> [105] "Non-ferrous metals"
#> [106] "Non-metallic minerals"
#> [107] "Paper, pulp and printing"
#> [108] "Stock changes [of Crude oil]"
#> [109] "Textile and leather"
#> [110] "Transport equipment"
#> [111] "Wood and wood products"
#> [112] "Stock changes [of Patent fuel]"
#> [113] "Imports [of Brown coal]"
#> [114] "Exports [of Brown coal]"
#> [115] "Imports [of Natural gas liquids]"
#> [116] "Imports [of Refinery feedstocks]"
#> [117] "Exports [of Natural gas liquids]"
#> [118] "Stock changes [of Natural gas liquids]"
#> [119] "Stock changes [of Refinery feedstocks]"
#> [120] "Imports [of Aviation gasoline]"
#> [121] "Imports [of Gasoline type jet fuel]"
#> [122] "Imports [of Other bituminous coal]"
#> [123] "Imports [of White spirit & SBP]"
#> [124] "Resources [of Coking coal]"
#> [125] "Resources [of Lignite]"
#> [126] "Resources [of Other bituminous coal]"
#> [127] "Resources [of Other hydrocarbons]"
#> [128] "Resources [of Sub-bituminous coal]"
#> [129] "Stock changes [of Coking coal]"
#> [130] "Stock changes [of Gasoline type jet fuel]"
#> [131] "Stock changes [of Other bituminous coal]"
#> [132] "Exports [of Coking coal]"
#> [133] "Exports [of Lignite]"
#> [134] "Exports [of Other bituminous coal]"
#> [135] "Exports [of White spirit & SBP]"
#> [136] "Stock changes [of Lignite]"
#> [137] "Stock changes [of White spirit & SBP]"
#> [138] "Exports [of Gasoline type jet fuel]"
#> [139] "Stock changes [of Sub-bituminous coal]"
#> [140] "Main activity producer CHP plants"
#> [141] "Resources [of Solar photovoltaics]"
#> [142] "Stock changes [of Other hydrocarbons]"
#> [143] "Manufacture [of Solar photovoltaics]"
#> [144] "Construction"
#> [145] "Imports [of Petroleum coke]"
#> [146] "Resources [of Solar thermal]"
#> [147] "Resources [of Wind]"
#> [148] "Manufacture [of Solar thermal]"
#> [149] "Manufacture [of Wind]"
#> [150] "Imports [of Other hydrocarbons]"
#> [151] "Resources [of Biogases]"
#> [152] "Resources [of Industrial waste]"
#> [153] "Resources [of Municipal waste (non-renewable)]"
#> [154] "Resources [of Municipal waste (renewable)]"
#> [155] "Resources [of Primary solid biofuels]"
#> [156] "Stock changes [of Ethane]"
#> [157] "For blended natural gas"
#> [158] "Manufacture [of Biogases]"
#> [159] "Manufacture [of Industrial waste]"
#> [160] "Manufacture [of Municipal waste (non-renewable)]"
#> [161] "Manufacture [of Municipal waste (renewable)]"
#> [162] "Manufacture [of Primary solid biofuels]"
#> [163] "Transport not elsewhere specified"
#> [164] "Imports [of Sub-bituminous coal]"
#> [165] "Imports [of Ethane]"
#> [166] "Resources [of Peat]"
#> [167] "Main activity producer heat plants"
#> [168] "Manufacture [of Peat]"
#> [169] "Imports [of Additives/blending components]"
#> [170] "Imports [of Biogasoline]"
#> [171] "Resources [of Additives/blending components]"
#> [172] "Resources [of Biogasoline]"
#> [173] "Manufacture [of Biogasoline]"
#> [174] "Exports [of Sub-bituminous coal]"
#> [175] "Stock changes [of Additives/blending components]"
#> [176] "Stock changes [of Biogasoline]"
#> [177] "Exports [of Additives/blending components]"
#> [178] "Imports [of Coking coal]"
#> [179] "Imports [of Lignite]"
#> [180] "Resources [of Biodiesels]"
#> [181] "Manufacture [of Biodiesels]"
#> [182] "Resources [of Other liquid biofuels]"
#> [183] "Manufacture [of Other liquid biofuels]"
#> [184] "Imports [of Anthracite]"
#> [185] "Resources [of Anthracite]"
#> [186] "Stock changes [of Anthracite]"
#> [187] "Exports [of Anthracite]"
#> [188] "Imports [of Biodiesels]"
#> [189] "Exports [of Biodiesels]"
#> [190] "Stock changes [of Biodiesels]"
#> [191] "Exports [of Biogasoline]"
#> [192] "Non-specified"
#> [193] "Non-energy use in other"
#> [194] "Exports to World marine bunkers [of Biodiesels]"
#> [195] "Exports [of Refinery feedstocks]"
#> [196] "Exports [of Ethane]"
#> [197] "Air compressors -> HPA"
#> [198] "Blast oxygen furnaces -> HTH.1600.C"
#> [199] "Coal stoves -> LTH.20.C"
#> [200] "Coal stoves -> LTH.50.C"
#> [201] "Coal stoves -> MTH.100.C"
#> [202] "Coal stoves -> MTH.200.C"
#> [203] "Commercial electric ovens -> MTH.200.C"
#> [204] "Commercial gas ovens -> MTH.200.C"
#> [205] "Commercial indoor electric lamps -> L"
#> [206] "Commercial outdoor electric lamps -> L"
#> [207] "Computers -> IP"
#> [208] "Diesel HDVs -> RoP"
#> [209] "Diesel trains -> RaP"
#> [210] "Domestic aircraft -> AirP"
#> [211] "Domestic appliances -> MD"
#> [212] "Domestic boats -> MaP"
#> [213] "Domestic electric lamps -> L"
#> [214] "Domestic electric ovens -> MTH.200.C"
#> [215] "Domestic gas ovens -> MTH.200.C"
#> [216] "Electric arc furnaces -> HTH.1600.C"
#> [217] "Electric cars -> RoP"
#> [218] "Electric pumps -> HPL"
#> [219] "Electric space heaters -> LTH.20.C"
#> [220] "Electric trains -> RaP"
#> [221] "Electric water heaters -> MTH.100.C"
#> [222] "Electro-chemical processors -> HTH.960.C"
#> [223] "Gas pump jacks -> HPL"
#> [224] "Gas space heaters -> LTH.20.C"
#> [225] "Gas water heaters -> MTH.100.C"
#> [226] "Gasoline LDVs -> RoP"
#> [227] "Gasoline motorcycles -> RoP"
#> [228] "HVAC -> LTC.20.C"
#> [229] "Industrial electric lamps -> L"
#> [230] "Industrial electric process heaters -> HTH.600.C"
#> [231] "Industrial fans -> KE"
#> [232] "Industrial refrigerators -> LTC.-10.C"
#> [233] "Kerosene stoves -> MTH.100.C"
#> [234] "LPG stoves -> LTH.20.C"
#> [235] "LPG stoves -> LTH.50.C"
#> [236] "LPG stoves -> MTH.100.C"
#> [237] "Machine tools -> MacW"
#> [238] "Material handlers -> MacW"
#> [239] "Natural gas vehicles -> RoP"
#> [240] "Non-energy consumption -> NEU"
#> [241] "Oil space heaters -> LTH.20.C"
#> [242] "Oil water heaters -> MTH.100.C"
#> [243] "Refrigerators -> LTC.-10.C"
#> [244] "Room air conditioners -> LTC.20.C"
#> [245] "Small scale biomass boilers -> HTH.1300.C"
#> [246] "Static diesel engines -> MD"
#> [247] "Static gasoline engines -> MD"
#> [248] "Steam cracking furnaces -> HTH.850.C"
#> [249] "Steam trains -> RaP"
#> [250] "Tractors -> MacW"
#> [251] "Wood stoves -> MTH.100.C"
#> [252] "Electric cable shovels -> MacW"
#> [253] "Electric continuous miners -> MF"
#> [254] "Ore grinding mills -> MF"
#> [255] "Pipeline gas compressors -> HPNG"
#> [256] "Ammonia synthesizers -> HTH.400.C"
#> [257] "Bayer kilns -> MTH.200.C"
#> [258] "Clinker kilns -> HTH.1000.C"
#> [259] "Electrolytic smelters -> HTH.960.C"
#> [260] "Paper machines -> MacW"
#> [261] "Heat machines -> MTH.300.C"
#> [262] "Diesel bulldozers -> MacW"
#> [263] "Diesel haulage trucks -> RoP"
#> [264] "Diesel hydraulic mobiles -> MacW"
#> [265] "Ground source heat pumps -> LTH.20.C"
#> [266] "Wood stoves -> LTH.20.C"
#> [267] "Wood stoves -> LTH.50.C"
#> [268] "Solar thermal water heaters -> LTH.60.C"
#> [269] "Electric cable cranes -> MacW"
#> [270] "Electric hand tools -> MacW"
#> [271] "Electric hydraulic mobiles -> MacW"
#> [272] "Diesel pumps -> HPL"
#> [273] "Diesel rock borers -> MF"
#> [274] "Resources [of Biomass]"
#> [275] "Farms"
#> [276] "Feed production"
#> [277] "Food production"
#> [278] "Asses -> AnMech"
#> [279] "Buffaloes -> AnMech"
#> [280] "Camelids -> AnMech"
#> [281] "Cattle -> AnMech"
#> [282] "Horses -> AnMech"
#> [283] "Human females -> HuMech"
#> [284] "Human males -> HuMech"
#> [285] "Mules -> AnMech"Aggregate by parts of row and column labels
Recca::get_all_products_and_industries() can also select
only parts of the row and column label. Knowing that we (eventually)
want to aggregate by the nouns, we can extract only the nouns (prefixes)
from the row and column labels, as shown by the code below. Extracting
by a piece of the row and column names is significantly slower than
extracting the entire row or column names.
df_with_prods_inds_nouns <- df |>
# Restrict years to reduce execution time.
dplyr::filter(Year == 1960) |>
Recca::get_all_products_and_industries(
piece = "noun",
inf_notation = TRUE,
notation = list(RCLabels::bracket_notation, RCLabels::arrow_notation))
dplyr::glimpse(df_with_prods_inds_nouns)
#> Rows: 12
#> Columns: 16
#> $ Country <chr> "USA", "USA", "USA", "USA", "USA", "USA", "USA", "USA",…
#> $ Method <chr> "PCM", "PCM", "PCM", "PCM", "PCM", "PCM", "PCM", "PCM",…
#> $ Energy.type <chr> "E", "E", "X", "X", "E", "E", "X", "X", "E", "E", "X", …
#> $ Last.stage <chr> "Final", "Useful", "Final", "Useful", "Final", "Useful"…
#> $ Year <dbl> 1960, 1960, 1960, 1960, 1960, 1960, 1960, 1960, 1960, 1…
#> $ IEAMW <chr> "IEA", "IEA", "IEA", "IEA", "MW", "MW", "MW", "MW", "Bo…
#> $ Y <list> <<matrix[24 x 43]>>, <<matrix[77 x 43]>>, <<matrix[24 …
#> $ S_units <list> <<matrix[36 x 1]>>, <<matrix[36 x 1]>>, <<matrix[36 x …
#> $ R <list> <<matrix[33 x 26]>>, <<matrix[33 x 26]>>, <<matrix[33 …
#> $ U <list> <<matrix[26 x 15]>>, <<matrix[53 x 70]>>, <<matrix[26 …
#> $ U_feed <list> <<matrix[21 x 15]>>, <<matrix[36 x 70]>>, <<matrix[21 …
#> $ U_EIOU <list> <<matrix[8 x 6]>>, <<matrix[17 x 6]>>, <<matrix[8 x 6]…
#> $ r_EIOU <list> <<matrix[26 x 15]>>, <<matrix[53 x 70]>>, <<matrix[26 …
#> $ V <list> <<matrix[15 x 28]>>, <<matrix[70 x 83]>>, <<matrix[15 …
#> $ Product.names <list> <"Bitumen", "Brown coal (if no detail)", "Coke oven co…
#> $ Industry.names <list> <"Imports", "Resources", "Statistical differences", "S…Again, one can use unique() to find the nouns required
for aggregation maps. These lists are shorter than lists of whole names
of products and industries, because there are fewer unique nouns than
unique row and column labels.
# Unique products (nouns only) in the data frame
df_with_prods_inds_nouns$Product.names |>
unlist() |>
unique()
#> [1] "Bitumen"
#> [2] "Brown coal (if no detail)"
#> [3] "Coke oven coke"
#> [4] "Crude/NGL/feedstocks (if no detail)"
#> [5] "Electricity"
#> [6] "Fuel oil"
#> [7] "Gas/diesel oil excl. biofuels"
#> [8] "Geothermal"
#> [9] "Hard coal (if no detail)"
#> [10] "Hydro"
#> [11] "Kerosene type jet fuel excl. biofuels"
#> [12] "Liquefied petroleum gases (LPG)"
#> [13] "Lubricants"
#> [14] "Motor gasoline excl. biofuels"
#> [15] "Natural gas"
#> [16] "Non-specified primary biofuels and waste"
#> [17] "Nuclear"
#> [18] "Other kerosene"
#> [19] "Other oil products"
#> [20] "Paraffin waxes"
#> [21] "Patent fuel"
#> [22] "Petroleum coke"
#> [23] "Blast furnace gas"
#> [24] "Coke oven gas"
#> [25] "Gas coke"
#> [26] "Gas works gas"
#> [27] "Refinery gas"
#> [28] "Aviation gasoline"
#> [29] "HPA"
#> [30] "HPL"
#> [31] "HTH.1600.C"
#> [32] "HTH.600.C"
#> [33] "HTH.850.C"
#> [34] "HTH.960.C"
#> [35] "IP"
#> [36] "KE"
#> [37] "L"
#> [38] "LTC.-10.C"
#> [39] "LTC.20.C"
#> [40] "LTH.20.C"
#> [41] "MacW"
#> [42] "MTH.100.C"
#> [43] "RoP"
#> [44] "AirP"
#> [45] "HTH.1300.C"
#> [46] "LTH.50.C"
#> [47] "MaP"
#> [48] "MD"
#> [49] "MTH.200.C"
#> [50] "NEU"
#> [51] "RaP"
#> [52] "Biomass"
#> [53] "Feed"
#> [54] "Food"
#> [55] "AnMech"
#> [56] "HuMech"
# Unique industries (nouns only) in the data frame
df_with_prods_inds_nouns$Industry.names |>
unlist() |>
unique()
#> [1] "Imports"
#> [2] "Resources"
#> [3] "Statistical differences"
#> [4] "Stock changes"
#> [5] "Blast furnaces"
#> [6] "Coal mines"
#> [7] "Coke ovens"
#> [8] "Gas works"
#> [9] "Main activity producer electricity plants"
#> [10] "Manufacture"
#> [11] "Natural gas extraction"
#> [12] "Nuclear industry"
#> [13] "Oil extraction"
#> [14] "Oil refineries"
#> [15] "Patent fuel plants"
#> [16] "Transfers"
#> [17] "Agriculture/forestry"
#> [18] "Chemical and petrochemical"
#> [19] "Commercial and public services"
#> [20] "Domestic aviation"
#> [21] "Domestic navigation"
#> [22] "Exports"
#> [23] "Exports to World aviation bunkers"
#> [24] "Exports to World marine bunkers"
#> [25] "Final consumption not elsewhere specified"
#> [26] "Industry not elsewhere specified"
#> [27] "Iron and steel"
#> [28] "Losses"
#> [29] "Non-energy use in transport"
#> [30] "Non-energy use industry/transformation/energy"
#> [31] "Rail"
#> [32] "Residential"
#> [33] "Road"
#> [34] "Air compressors"
#> [35] "Blast oxygen furnaces"
#> [36] "Coal stoves"
#> [37] "Commercial electric ovens"
#> [38] "Commercial gas ovens"
#> [39] "Commercial indoor electric lamps"
#> [40] "Commercial outdoor electric lamps"
#> [41] "Computers"
#> [42] "Diesel HDVs"
#> [43] "Diesel trains"
#> [44] "Domestic aircraft"
#> [45] "Domestic appliances"
#> [46] "Domestic boats"
#> [47] "Domestic electric lamps"
#> [48] "Domestic electric ovens"
#> [49] "Domestic gas ovens"
#> [50] "Electric arc furnaces"
#> [51] "Electric cars"
#> [52] "Electric pumps"
#> [53] "Electric space heaters"
#> [54] "Electric trains"
#> [55] "Electric water heaters"
#> [56] "Electro-chemical processors"
#> [57] "Gas pump jacks"
#> [58] "Gas space heaters"
#> [59] "Gas water heaters"
#> [60] "Gasoline LDVs"
#> [61] "Gasoline motorcycles"
#> [62] "HVAC"
#> [63] "Industrial electric lamps"
#> [64] "Industrial electric process heaters"
#> [65] "Industrial fans"
#> [66] "Industrial refrigerators"
#> [67] "Kerosene stoves"
#> [68] "LPG stoves"
#> [69] "Machine tools"
#> [70] "Material handlers"
#> [71] "Natural gas vehicles"
#> [72] "Non-energy consumption"
#> [73] "Oil space heaters"
#> [74] "Oil water heaters"
#> [75] "Refrigerators"
#> [76] "Room air conditioners"
#> [77] "Small scale biomass boilers"
#> [78] "Static diesel engines"
#> [79] "Static gasoline engines"
#> [80] "Steam cracking furnaces"
#> [81] "Steam trains"
#> [82] "Tractors"
#> [83] "Wood stoves"
#> [84] "Farms"
#> [85] "Feed production"
#> [86] "Food production"
#> [87] "Transport not elsewhere specified"
#> [88] "Asses"
#> [89] "Buffaloes"
#> [90] "Camelids"
#> [91] "Cattle"
#> [92] "Horses"
#> [93] "Human females"
#> [94] "Human males"
#> [95] "Mules"The lists of nouns can be used to create aggregation maps, as shown in the next section.
Construct an aggregation map
An aggregation map is a description of the way aggregation is to be
performed. In R terms, an aggregation map is simply a named
list where values are the row or column names (or pieces of names) to be
aggregated and names are the new names for the aggregated rows or
columns.
For example,
prod_agg_map_example <- list(
`Coal & coal products` = c("Hard coal (if no detail)",
"Brown coal (if no detail)",
"Anthracite",
"Coking coal",
"Other bituminous coal",
"Sub-bituminous coal"),
`Oil & oil products` = c("Crude/NGL/feedstocks (if no detail)",
"Crude oil",
"Natural gas liquids",
"Refinery feedstocks",
"Additives/blending components",
"Other hydrocarbons")
)Aggregation maps can also be constructed from a data frame with “Many” and “Few” columns.
prod_agg_table_example <- tibble::tribble(
~Many, ~Few,
"Hard coal (if no detail)", "Coal & coal products",
"Brown coal (if no detail)", "Coal & coal products",
"Anthracite", "Coal & coal products",
"Coking coal", "Coal & coal products",
"Other bituminous coal", "Coal & coal products",
"Sub-bituminous coal", "Coal & coal products",
"Crude/NGL/feedstocks (if no detail)", "Oil & oil products",
"Crude oil", "Oil & oil products",
"Natural gas liquids", "Oil & oil products",
"Refinery feedstocks", "Oil & oil products",
"Additives/blending components", "Oil & oil products",
"Other hydrocarbons", "Oil & oil products"
)
prod_agg_table_example
#> # A tibble: 12 × 2
#> Many Few
#> <chr> <chr>
#> 1 Hard coal (if no detail) Coal & coal products
#> 2 Brown coal (if no detail) Coal & coal products
#> 3 Anthracite Coal & coal products
#> 4 Coking coal Coal & coal products
#> 5 Other bituminous coal Coal & coal products
#> 6 Sub-bituminous coal Coal & coal products
#> 7 Crude/NGL/feedstocks (if no detail) Oil & oil products
#> 8 Crude oil Oil & oil products
#> 9 Natural gas liquids Oil & oil products
#> 10 Refinery feedstocks Oil & oil products
#> 11 Additives/blending components Oil & oil products
#> 12 Other hydrocarbons Oil & oil products
matsbyname::agg_table_to_agg_map(prod_agg_table_example,
many_colname = "Many",
few_colname = "Few")
#> $`Coal & coal products`
#> [1] "Hard coal (if no detail)" "Brown coal (if no detail)"
#> [3] "Anthracite" "Coking coal"
#> [5] "Other bituminous coal" "Sub-bituminous coal"
#>
#> $`Oil & oil products`
#> [1] "Crude/NGL/feedstocks (if no detail)" "Crude oil"
#> [3] "Natural gas liquids" "Refinery feedstocks"
#> [5] "Additives/blending components" "Other hydrocarbons"Aggregation tables can be created in an Excel file and read into
R with the readxl package. Example aggregation
tables can be found in the “Aggregation_maps.xlsx” file in the
PFUPipeline package.
prod_agg_map <- system.file("extdata", "Aggregation_Tables.xlsx",
package="PFUPipeline") |>
readxl::read_excel(sheet = "product_aggregation") |>
matsbyname::agg_table_to_agg_map(many_colname = "Many",
few_colname = "Few")
prod_agg_map
#> $Biofuels
#> [1] "Primary solid biofuels"
#> [2] "Biogases"
#> [3] "Biogasoline"
#> [4] "Biodiesels"
#> [5] "Bio jet kerosene"
#> [6] "Other liquid biofuels"
#> [7] "Non-specified primary biofuels and waste"
#> [8] "Charcoal"
#>
#> $`Coal & coal products`
#> [1] "Hard coal (if no detail)" "Brown coal (if no detail)"
#> [3] "Anthracite" "Coking coal"
#> [5] "Other bituminous coal" "Sub-bituminous coal"
#> [7] "Lignite" "Patent fuel"
#> [9] "Coke oven coke" "Gas coke"
#> [11] "Coal tar" "BKB"
#> [13] "Gas works gas" "Coke oven gas"
#> [15] "Blast furnace gas" "Other recovered gases"
#> [17] "Peat" "Peat products"
#>
#> $Electricity
#> [1] "Nuclear" "Hydro" "Solar photovoltaics"
#> [4] "Tide, wave and ocean" "Wind" "Other sources"
#> [7] "Electricity"
#>
#> $Feed
#> [1] "Feed"
#>
#> $Food
#> [1] "Food"
#>
#> $HPF
#> [1] "HPA" "HPL" "HPNG"
#>
#> $HTH
#> [1] "HTH.1000.C" "HTH.1300.C" "HTH.1600.C" "HTH.400.C" "HTH.600.C"
#> [6] "HTH.850.C" "HTH.960.C"
#>
#> $Heat
#> [1] "Elec/heat output from non-specified manufactured gases"
#> [2] "Heat output from non-specified combustible fuels"
#> [3] "Geothermal"
#> [4] "Solar thermal"
#> [5] "Heat"
#>
#> $IP
#> [1] "IP"
#>
#> $L
#> [1] "L"
#>
#> $LTC
#> [1] "LTC.-10.C" "LTC.20.C"
#>
#> $LTH
#> [1] "LTH.20.C" "LTH.50.C" "LTH.60.C"
#>
#> $MTH
#> [1] "MTH.100.C" "MTH.200.C" "MTH.300.C"
#>
#> $MechW
#> [1] "KE" "MD" "MF" "MacW"
#>
#> $MuscW
#> [1] "AMW" "HMW"
#>
#> $NEU
#> [1] "NEU"
#>
#> $`Natural gas`
#> [1] "Natural gas"
#>
#> $`Oil & oil products`
#> [1] "Oil shale and oil sands"
#> [2] "Crude/NGL/feedstocks (if no detail)"
#> [3] "Crude oil"
#> [4] "Natural gas liquids"
#> [5] "Refinery feedstocks\""
#> [6] "Additives/blending components"
#> [7] "Other hydrocarbons"
#> [8] "Refinery gas"
#> [9] "Ethane"
#> [10] "Liquefied petroleum gases (LPG)"
#> [11] "Motor gasoline excl. biofuels"
#> [12] "Aviation gasoline"
#> [13] "Gasoline type jet fuel"
#> [14] "Kerosene type jet fuel excl. biofuels"
#> [15] "Other kerosene"
#> [16] "Gas/diesel oil excl. biofuels"
#> [17] "Fuel oil"
#> [18] "Naphtha"
#> [19] "White spirit & SBP"
#> [20] "Lubricants"
#> [21] "Bitumen"
#> [22] "Paraffin waxes"
#> [23] "Petroleum coke"
#> [24] "Other oil products"
#>
#> $Prop
#> [1] "AirP" "MaP" "RaP" "RoP"
#>
#> $Waste
#> [1] "Industrial waste" "Municipal waste (renewable)"
#> [3] "Municipal waste (non-renewable)"
ind_agg_map <- system.file("extdata", "Aggregation_Tables.xlsx",
package="PFUPipeline") |>
readxl::read_excel(sheet = "industry_aggregation") |>
matsbyname::agg_table_to_agg_map(many_colname = "Many",
few_colname = "Few")
ind_agg_map
#> $`Agriculture, forestry, and fishing`
#> [1] "Agriculture/forestry" "Fishing"
#>
#> $`Commercial and public services`
#> [1] "Commercial and public services"
#>
#> $`Energy industry own use`
#> [1] "Coal mines"
#> [2] "Oil and gas extraction"
#> [3] "Blast furnaces"
#> [4] "Gas works"
#> [5] "Gasification plants for biogases"
#> [6] "Coke ovens"
#> [7] "Patent fuel plants"
#> [8] "BKB/peat briquette plants"
#> [9] "Oil refineries"
#> [10] "Coal liquefaction plants"
#> [11] "Liquefaction (LNG) / regasification plants"
#> [12] "Gas-to-liquids (GTL) plants"
#> [13] "Own use in electricity, CHP and heat plants"
#> [14] "Pumped storage plants"
#> [15] "Nuclear industry"
#> [16] "Charcoal production plants"
#> [17] "Non-specified (energy)"
#> [18] "Main activity producer electricity plants"
#> [19] "Main activity producer CHP plants"
#> [20] "Main activity producer heat plants"
#> [21] "Manufacture"
#> [22] "Natural gas extraction"
#> [23] "Oil extraction"
#> [24] "Transfers"
#>
#> $Industry
#> [1] "Mining and quarrying" "Construction"
#> [3] "Iron and steel" "Chemical and petrochemical"
#> [5] "Non-ferrous metals" "Non-metallic minerals"
#> [7] "Transport equipment" "Machinery"
#> [9] "Food and tobacco" "Paper, pulp and printing"
#> [11] "Wood and wood products" "Textile and leather"
#> [13] "Industry not elsewhere specified"
#>
#> $Manufacture
#> [1] "Manufacture"
#>
#> $`Non-energy use`
#> [1] "Non-energy use industry/transformation/energy"
#> [2] "Non-energy use in transport"
#> [3] "Non-energy use in other"
#> [4] "Non-energy use in construction"
#> [5] "Non-energy use in mining and quarrying"
#> [6] "Non-energy use in iron and steel"
#> [7] "Non-energy use in chemical/petrochemical"
#> [8] "Non-energy use in non-ferrous metals"
#> [9] "Non-energy use in non-metallic minerals"
#> [10] "Non-energy use in transport equipment"
#> [11] "Non-energy use in machinery"
#> [12] "Non-energy use in food/beverages/tobacco"
#> [13] "Non-energy use in paper/pulp and printing"
#> [14] "Non-energy use in wood and wood products"
#> [15] "Non-energy use in textiles and leather"
#> [16] "Non-energy use in industry not elsewhere specified"
#>
#> $Residential
#> [1] "Residential"
#>
#> $Transport
#> [1] "World aviation bunkers" "Domestic aviation"
#> [3] "Road" "Rail"
#> [5] "Pipeline transport" "World marine bunkers"
#> [7] "Exports to World marine bunkers" "Domestic navigation"
#> [9] "Transport not elsewhere specified" "International aviation"
#> [11] "International navigation"Perform the aggregations
There are two matsbyname functions that assist with
aggregations: matsbyname::aggregate_byname() and
matsbyname::aggregate_pieces_byname(). A third function
(matsbyname::aggregate_to_pref_suff_byname()) has been
deprecated in favor of
matsbyname::aggregate_pieces_byname().
Knowing that the matsbyname package can aggregate by any
piece of a row or column label (including prefix, suffix, noun, or
object of a preposition), aggregation maps can be constructed by
focusing only on the nouns (prefixes) of the row and column names.
To aggregate the matrices in the df object, one can use
code like the following example.
aggregated <- df |>
# Restrict to a single year to reduce execution time.
dplyr::filter(Year == 1960) |>
dplyr::mutate(
# Aggregate R
R_agg = .data[["R"]] |>
matsbyname::aggregate_pieces_byname(
piece = "noun",
margin = "Industry",
inf_notation = TRUE,
notation = list(list(RCLabels::bracket_notation, RCLabels::arrow_notation)),
) |>
matsbyname::aggregate_pieces_byname(
piece = "noun",
margin = "Product",
inf_notation = FALSE,
notation = RCLabels::bracket_notation,
aggregation_map = list(prod_agg_map)
),
# Aggregate U_feed
U_feed_agg = .data[["U_feed"]] |>
matsbyname::aggregate_pieces_byname(
piece = "noun",
margin = "Product",
inf_notation = FALSE,
notation = RCLabels::bracket_notation,
aggregation_map = list(prod_agg_map)
) |>
matsbyname::aggregate_pieces_byname(
piece = "noun",
margin = "Industry",
inf_notation = TRUE,
notation = list(list(RCLabels::bracket_notation, RCLabels::arrow_notation)),
aggregation_map = list(ind_agg_map)
),
# Aggregate U_EIOU
U_EIOU_agg = .data[["U_EIOU"]] |>
matsbyname::aggregate_pieces_byname(
piece = "noun",
margin = "Product",
inf_notation = FALSE,
notation = RCLabels::bracket_notation,
aggregation_map = list(prod_agg_map)
) |>
matsbyname::aggregate_pieces_byname(
piece = "noun",
margin = "Industry",
inf_notation = TRUE,
notation = list(list(RCLabels::bracket_notation, RCLabels::arrow_notation)),
aggregation_map = list(ind_agg_map)
),
# Create aggregated U matrix
U_agg = matsbyname::sum_byname(.data[["U_feed_agg"]], .data[["U_EIOU_agg"]]),
# Create aggregated r_EIOU matrix
r_EIOU_agg = matsbyname::quotient_byname(.data[["U_EIOU"]], .data[["U"]]) |>
matsbyname::replaceNaN_byname(),
# Aggregate V
V_agg = .data[["V"]] |>
matsbyname::aggregate_pieces_byname(
piece = "noun",
margin = "Industry",
inf_notation = TRUE,
notation = list(list(RCLabels::bracket_notation, RCLabels::arrow_notation)),
) |>
matsbyname::aggregate_pieces_byname(
piece = "noun",
margin = "Product",
inf_notation = FALSE,
notation = RCLabels::bracket_notation,
aggregation_map = list(prod_agg_map)
),
# Aggregate Y
Y_agg = .data[["Y"]] |>
matsbyname::aggregate_pieces_byname(
piece = "noun",
margin = "Product",
inf_notation = FALSE,
notation = RCLabels::bracket_notation,
aggregation_map = list(prod_agg_map)
) |>
matsbyname::aggregate_pieces_byname(
piece = "noun",
margin = "Industry",
inf_notation = TRUE,
notation = list(list(RCLabels::bracket_notation, RCLabels::arrow_notation)),
aggregation_map = list(ind_agg_map)
)
)The aggregated matrices are simpler than the original matrices, as expected.
aggregated$R_agg[[1]]
#> Biofuels Coal & coal products Electricity Heat
#> Imports 0 225 458 0
#> Resources 32367 243640 12823 30
#> Statistical differences 1 6571 0 0
#> Stock changes 0 771 0 0
#> Natural gas Oil & oil products
#> Imports 3656 93392
#> Resources 283514 392570
#> Statistical differences 1 7046
#> Stock changes 0 5999
#> attr(,"rowtype")
#> [1] "Industry"
#> attr(,"coltype")
#> [1] "Product"
aggregated$U_feed_agg[[1]]
#> Energy industry own use
#> Biofuels 32388
#> Coal & coal products 433824
#> Electricity 25646
#> Heat 60
#> Natural gas 327540
#> Oil & oil products 855297
#> attr(,"rowtype")
#> [1] "Product"
#> attr(,"coltype")
#> [1] "Industry"
aggregated$U_EIOU_agg[[1]]
#> Energy industry own use
#> Coal & coal products 6473
#> Electricity 3756
#> Natural gas 37137
#> Oil & oil products 35612
#> attr(,"rowtype")
#> [1] "Product"
#> attr(,"coltype")
#> [1] "Industry"
aggregated$U_agg[[1]]
#> Energy industry own use
#> Biofuels 32388
#> Coal & coal products 440297
#> Electricity 29402
#> Heat 60
#> Natural gas 364677
#> Oil & oil products 890909
#> attr(,"rowtype")
#> [1] "Product"
#> attr(,"coltype")
#> [1] "Industry"
aggregated$V_agg[[1]]
#> Biofuels Coal & coal products
#> Blast furnaces 0 9500
#> Coal mines 0 243640
#> Coke ovens 0 46862
#> Gas works 0 4365
#> Main activity producer electricity plants 0 0
#> Manufacture 32367 0
#> Natural gas extraction 0 0
#> Nuclear industry 0 0
#> Oil extraction 0 0
#> Oil refineries 0 0
#> Patent fuel plants 0 489
#> Transfers 0 0
#> Electricity Heat Natural gas
#> Blast furnaces 0.00 0 0
#> Coal mines 0.00 0 0
#> Coke ovens 0.00 0 0
#> Gas works 0.00 0 0
#> Main activity producer electricity plants 68712.48 0 0
#> Manufacture 12823.00 30 0
#> Natural gas extraction 0.00 0 283514
#> Nuclear industry 47.52 0 0
#> Oil extraction 0.00 0 0
#> Oil refineries 0.00 0 0
#> Patent fuel plants 0.00 0 0
#> Transfers 0.00 0 0
#> Oil & oil products
#> Blast furnaces 0
#> Coal mines 0
#> Coke ovens 0
#> Gas works 0
#> Main activity producer electricity plants 0
#> Manufacture 0
#> Natural gas extraction 0
#> Nuclear industry 0
#> Oil extraction 392570
#> Oil refineries 439718
#> Patent fuel plants 0
#> Transfers 15901
#> attr(,"rowtype")
#> [1] "Industry"
#> attr(,"coltype")
#> [1] "Product"
aggregated$Y_agg[[1]]
#> Agriculture, forestry, and fishing
#> Biofuels 0
#> Coal & coal products 0
#> Electricity 2517
#> Natural gas 0
#> Oil & oil products 8250
#> Commercial and public services Exports
#> Biofuels 0 0
#> Coal & coal products 13883 21586
#> Electricity 11032 68
#> Natural gas 23955 265
#> Oil & oil products 27606 10063
#> Exports to World aviation bunkers
#> Biofuels 0
#> Coal & coal products 0
#> Electricity 0
#> Natural gas 0
#> Oil & oil products 1895
#> Final consumption not elsewhere specified Industry Losses
#> Biofuels 15666 16681 0
#> Coal & coal products 0 65298 0
#> Electricity 0 27859 6237
#> Natural gas 0 94743 11090
#> Oil & oil products 9886 74409 0
#> Non-energy use Residential Statistical differences
#> Biofuels 0 0 0
#> Coal & coal products 0 11960 1530
#> Electricity 0 17283 0
#> Natural gas 0 72862 0
#> Oil & oil products 40462 45566 4
#> Stock changes Transport
#> Biofuels 0 0
#> Coal & coal products 33 1476
#> Electricity 0 466
#> Natural gas 3093 0
#> Oil & oil products 1618 236528
#> attr(,"rowtype")
#> [1] "Product"
#> attr(,"coltype")
#> [1] "Industry"Conclusion
Through judicious use of several helpful functions
(Recca::get_all_products_and_industries(),
matsbyname::agg_table_to_agg_map(), and
matsbyname::aggregate_pieces_byname()), one can aggregate
PSUT matrices when they are arranged in a matsindf data frame.