Energy Consumption Trends: Analyze the data to identify trends in global energy consumption over time. This can help governments and organizations make informed decisions about energy policies, investments, and resource allocation.
Renewable Energy Adoption: Determine the adoption rate of renewable energy sources (such as solar, wind, and hydropower) across different countries and regions. This information can be valuable for assessing progress toward sustainable energy goals.
Carbon Emissions Analysis: Investigate the relationship between energy production methods and carbon emissions. Python data analysis can help in identifying sources of high emissions and formulating strategies for reducing carbon footprints.
#Importing libraries
import pandas as pd
import numpy as np
import seaborn as sns
from datetime import datetime
import matplotlib as mpl
import matplotlib.pyplot as plt
import seaborn as sns
import pycountry_convert as pc
import plotly.graph_objects as go
import plotly.subplots as sp
import plotly.graph_objs as go
from sklearn.preprocessing import StandardScaler
from sklearn.preprocessing import MinMaxScaler
from sklearn.linear_model import LinearRegression
from sklearn.ensemble import RandomForestRegressor
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error, r2_score, mean_absolute_error
from sklearn import metrics
from math import sqrt
from statsmodels.stats.outliers_influence import variance_inflation_factor
from statsmodels.tools.tools import add_constant
from sklearn.cluster import KMeans
pd.set_option('display.max_columns', None)
%matplotlib inline
#Loading,reading and storing dataset
df = pd.read_csv('global_data.csv')
df.head()
Entity | Year | Access to electricity (% of population) | Access to clean fuels for cooking | Renewable-electricity-generating-capacity-per-capita | Financial flows to developing countries (US $) | Renewable energy share in the total final energy consumption (%) | Electricity from fossil fuels (TWh) | Electricity from nuclear (TWh) | Electricity from renewables (TWh) | Low-carbon electricity (% electricity) | Primary energy consumption per capita (kWh/person) | Energy intensity level of primary energy (MJ/$2017 PPP GDP) | Value_co2_emissions_kt_by_country | Renewables (% equivalent primary energy) | gdp_growth | gdp_per_capita | Density\n(P/Km2) | Land Area(Km2) | Latitude | Longitude | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | Afghanistan | 2000 | 1.613591 | 6.2 | 9.22 | 20000.0 | 44.99 | 0.16 | 0.0 | 0.31 | 65.957440 | 302.59482 | 1.64 | 760.000000 | NaN | NaN | NaN | 60 | 652230.0 | 33.93911 | 67.709953 |
1 | Afghanistan | 2001 | 4.074574 | 7.2 | 8.86 | 130000.0 | 45.60 | 0.09 | 0.0 | 0.50 | 84.745766 | 236.89185 | 1.74 | 730.000000 | NaN | NaN | NaN | 60 | 652230.0 | 33.93911 | 67.709953 |
2 | Afghanistan | 2002 | 9.409158 | 8.2 | 8.47 | 3950000.0 | 37.83 | 0.13 | 0.0 | 0.56 | 81.159424 | 210.86215 | 1.40 | 1029.999971 | NaN | NaN | 179.426579 | 60 | 652230.0 | 33.93911 | 67.709953 |
3 | Afghanistan | 2003 | 14.738506 | 9.5 | 8.09 | 25970000.0 | 36.66 | 0.31 | 0.0 | 0.63 | 67.021280 | 229.96822 | 1.40 | 1220.000029 | NaN | 8.832278 | 190.683814 | 60 | 652230.0 | 33.93911 | 67.709953 |
4 | Afghanistan | 2004 | 20.064968 | 10.9 | 7.75 | NaN | 44.24 | 0.33 | 0.0 | 0.56 | 62.921350 | 204.23125 | 1.20 | 1029.999971 | NaN | 1.414118 | 211.382074 | 60 | 652230.0 | 33.93911 | 67.709953 |
Entity: The name of the country or region for which the data is reported.
Year: The year for which the data is reported, ranging from 2000 to 2020.
Access to electricity (% of population): The percentage of population with access to electricity.
Access to clean fuels for cooking (% of population): The percentage of the population with primary reliance on clean fuels.
Renewable-electricity-generating-capacity-per-capita: Installed Renewable energy capacity per person
Financial flows to developing countries (US): Aid and assistance from developed countries for clean energy projects.
Renewable energy share in total final energy consumption (%): Percentage of renewable energy in final energy consumption.
Electricity from fossil fuels (TWh): Electricity generated from fossil fuels (coal, oil, gas) in terawatt-hours.
Electricity from nuclear (TWh): Electricity generated from nuclear power in terawatt-hours.
Electricity from renewables (TWh): Electricity generated from renewable sources (hydro, solar, wind, etc.) in terawatt-hours.
Low-carbon electricity (% electricity): Percentage of electricity from low-carbon sources (nuclear and renewables).
Primary energy consumption per capita (kWh/person): Energy consumption per person in kilowatt-hours.
Energy intensity level of primary energy (MJ/$2011 PPP GDP): Energy use per unit of GDP at purchasing power parity.
Value_co2_emissions (metric tons per capita): Carbon dioxide emissions per person in metric tons.
Renewables(% equivalent primary energy): Equivalent primary energy that is derived from renewable sources.
GDP growth (annual %): Annual GDP growth rate based on constant local currency.
GDP per capita: Gross domestic product per person.
Density (P/Km2): Population density in persons per square kilometer.
Land Area (Km2): Total land area in square kilometers.
Latitude: Latitude of the country's centroid in decimal degrees.
Longitude: Longitude of the country's centroid in decimal degrees.
#Show the DataFrame's shape
df.shape
(3649, 21)
# Standardize column names
df.columns=[e.lower().replace(' ', '').replace('-', '_') for e in df.columns]
df.columns
Index(['entity', 'year', 'accesstoelectricity(%ofpopulation)', 'accesstocleanfuelsforcooking', 'renewable_electricity_generating_capacity_per_capita', 'financialflowstodevelopingcountries(us$)', 'renewableenergyshareinthetotalfinalenergyconsumption(%)', 'electricityfromfossilfuels(twh)', 'electricityfromnuclear(twh)', 'electricityfromrenewables(twh)', 'low_carbonelectricity(%electricity)', 'primaryenergyconsumptionpercapita(kwh/person)', 'energyintensitylevelofprimaryenergy(mj/$2017pppgdp)', 'value_co2_emissions_kt_by_country', 'renewables(%equivalentprimaryenergy)', 'gdp_growth', 'gdp_per_capita', 'density\n(p/km2)', 'landarea(km2)', 'latitude', 'longitude'], dtype='object')
df = df.rename(columns = {"density\\n(p/km2)":"density(P/km2)",
"value_co2_emissions_kt_by_country":"CO2_emissions_value_by_country_kt",
"accesstocleanfuelsforcooking":"access_to_clean_fuels_for_cooking",
"financialflowstodevelopingcountries(us$)":"financial_flows_to_developing_countries(us$)",
"renewableenergyshareinthetotalfinalenergyconsumption(%)":"renewable_energy_share_in_the_total_final_energy_consumption(%)",
"electricityfromfossilfuels(twh)":"electricity_from_fossil_fuels(twh)",
"primaryenergyconsumptionpercapita(kwh/person)":"primary_energy_consumption_per_capita(kwh/person)",
"energyintensitylevelofprimaryenergy(mj/$2017pppgdp)":"energy_intensity_level_of_primary_energy(mj/$2017pppgdp)",
"renewables(%equivalentprimaryenergy)": "renewables(%equivalent_primary_energy)",
"accesstoelectricity(%ofpopulation)":"access_to_electricity(%_of_population)",
"electricityfromnuclear(twh)":"electricity_from_nuclear(twh)",
"electricityfromrenewables(twh)":"electricity_from_renewables(twh)"
})
df.head()
entity | year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | density(P/km2) | landarea(km2) | latitude | longitude | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | Afghanistan | 2000 | 1.613591 | 6.2 | 9.22 | 20000.0 | 44.99 | 0.16 | 0.0 | 0.31 | 65.957440 | 302.59482 | 1.64 | 760.000000 | NaN | NaN | NaN | 60 | 652230.0 | 33.93911 | 67.709953 |
1 | Afghanistan | 2001 | 4.074574 | 7.2 | 8.86 | 130000.0 | 45.60 | 0.09 | 0.0 | 0.50 | 84.745766 | 236.89185 | 1.74 | 730.000000 | NaN | NaN | NaN | 60 | 652230.0 | 33.93911 | 67.709953 |
2 | Afghanistan | 2002 | 9.409158 | 8.2 | 8.47 | 3950000.0 | 37.83 | 0.13 | 0.0 | 0.56 | 81.159424 | 210.86215 | 1.40 | 1029.999971 | NaN | NaN | 179.426579 | 60 | 652230.0 | 33.93911 | 67.709953 |
3 | Afghanistan | 2003 | 14.738506 | 9.5 | 8.09 | 25970000.0 | 36.66 | 0.31 | 0.0 | 0.63 | 67.021280 | 229.96822 | 1.40 | 1220.000029 | NaN | 8.832278 | 190.683814 | 60 | 652230.0 | 33.93911 | 67.709953 |
4 | Afghanistan | 2004 | 20.064968 | 10.9 | 7.75 | NaN | 44.24 | 0.33 | 0.0 | 0.56 | 62.921350 | 204.23125 | 1.20 | 1029.999971 | NaN | 1.414118 | 211.382074 | 60 | 652230.0 | 33.93911 | 67.709953 |
df.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 3649 entries, 0 to 3648 Data columns (total 21 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 entity 3649 non-null object 1 year 3649 non-null int64 2 access_to_electricity(%_of_population) 3639 non-null float64 3 access_to_clean_fuels_for_cooking 3480 non-null float64 4 renewable_electricity_generating_capacity_per_capita 2718 non-null float64 5 financial_flows_to_developing_countries(us$) 1560 non-null float64 6 renewable_energy_share_in_the_total_final_energy_consumption(%) 3455 non-null float64 7 electricity_from_fossil_fuels(twh) 3628 non-null float64 8 electricity_from_nuclear(twh) 3523 non-null float64 9 electricity_from_renewables(twh) 3628 non-null float64 10 low_carbonelectricity(%electricity) 3607 non-null float64 11 primary_energy_consumption_per_capita(kwh/person) 3649 non-null float64 12 energy_intensity_level_of_primary_energy(mj/$2017pppgdp) 3442 non-null float64 13 CO2_emissions_value_by_country_kt 3221 non-null float64 14 renewables(%equivalent_primary_energy) 1512 non-null float64 15 gdp_growth 3332 non-null float64 16 gdp_per_capita 3367 non-null float64 17 density(P/km2) 3648 non-null object 18 landarea(km2) 3648 non-null float64 19 latitude 3648 non-null float64 20 longitude 3648 non-null float64 dtypes: float64(18), int64(1), object(2) memory usage: 598.8+ KB
#df['year'] = pd.to_datetime(df['year'], format='%Y')
# Llena los valores faltantes con ceros
#df['density(P/km2)'].fillna(0, inplace=True)
#df['density(P/km2)'] = df['density(P/km2)'].str.replace(',', '', regex=True) # Elimina las comas, si las hay
#df['density(P/km2)'] = df['density(P/km2)'].astype(int)
#Checking for duplicate rows in the data to remove if any
df.duplicated().sum()
0
##hecking null values
df.isnull().sum().sort_values(ascending=False)
renewables(%equivalent_primary_energy) 2137 financial_flows_to_developing_countries(us$) 2089 renewable_electricity_generating_capacity_per_capita 931 CO2_emissions_value_by_country_kt 428 gdp_growth 317 gdp_per_capita 282 energy_intensity_level_of_primary_energy(mj/$2017pppgdp) 207 renewable_energy_share_in_the_total_final_energy_consumption(%) 194 access_to_clean_fuels_for_cooking 169 electricity_from_nuclear(twh) 126 low_carbonelectricity(%electricity) 42 electricity_from_renewables(twh) 21 electricity_from_fossil_fuels(twh) 21 access_to_electricity(%_of_population) 10 latitude 1 landarea(km2) 1 density(P/km2) 1 longitude 1 primary_energy_consumption_per_capita(kwh/person) 0 year 0 entity 0 dtype: int64
#Checking % of null values
null_percentage = [(col, 100 * sum(df[col].isnull()) / len(df[col])) for col in df.columns]
# Sort list of tuples by percentage of null values from largest to smallest
sorted_null_percentage = sorted(null_percentage, key=lambda x: x[1], reverse=True)
# Print the ordered list
print("Nulls (sorted by percentage descending):")
for col, percentage in sorted_null_percentage:
print(f"{col} contains {percentage:.2f}%")
Nulls (sorted by percentage descending): renewables(%equivalent_primary_energy) contains 58.56% financial_flows_to_developing_countries(us$) contains 57.25% renewable_electricity_generating_capacity_per_capita contains 25.51% CO2_emissions_value_by_country_kt contains 11.73% gdp_growth contains 8.69% gdp_per_capita contains 7.73% energy_intensity_level_of_primary_energy(mj/$2017pppgdp) contains 5.67% renewable_energy_share_in_the_total_final_energy_consumption(%) contains 5.32% access_to_clean_fuels_for_cooking contains 4.63% electricity_from_nuclear(twh) contains 3.45% low_carbonelectricity(%electricity) contains 1.15% electricity_from_fossil_fuels(twh) contains 0.58% electricity_from_renewables(twh) contains 0.58% access_to_electricity(%_of_population) contains 0.27% density(P/km2) contains 0.03% landarea(km2) contains 0.03% latitude contains 0.03% longitude contains 0.03% entity contains 0.00% year contains 0.00% primary_energy_consumption_per_capita(kwh/person) contains 0.00%
# Number of zero values in the dataset (in percent)
missing_values_df = df.isnull().sum() / df.shape[0] * 100.00
print(round(missing_values_df),2)
entity 0.0 year 0.0 access_to_electricity(%_of_population) 0.0 access_to_clean_fuels_for_cooking 5.0 renewable_electricity_generating_capacity_per_capita 26.0 financial_flows_to_developing_countries(us$) 57.0 renewable_energy_share_in_the_total_final_energy_consumption(%) 5.0 electricity_from_fossil_fuels(twh) 1.0 electricity_from_nuclear(twh) 3.0 electricity_from_renewables(twh) 1.0 low_carbonelectricity(%electricity) 1.0 primary_energy_consumption_per_capita(kwh/person) 0.0 energy_intensity_level_of_primary_energy(mj/$2017pppgdp) 6.0 CO2_emissions_value_by_country_kt 12.0 renewables(%equivalent_primary_energy) 59.0 gdp_growth 9.0 gdp_per_capita 8.0 density(P/km2) 0.0 landarea(km2) 0.0 latitude 0.0 longitude 0.0 dtype: float64 2
There are two feautures with a rather large number of zero values:
Cleaning can cause significant errors in the study, as more than 57% of the data will be removed.
df.isnull().sum().sort_values(ascending=False)
renewables(%equivalent_primary_energy) 2137 financial_flows_to_developing_countries(us$) 2089 renewable_electricity_generating_capacity_per_capita 931 CO2_emissions_value_by_country_kt 428 gdp_growth 317 gdp_per_capita 282 energy_intensity_level_of_primary_energy(mj/$2017pppgdp) 207 renewable_energy_share_in_the_total_final_energy_consumption(%) 194 access_to_clean_fuels_for_cooking 169 electricity_from_nuclear(twh) 126 low_carbonelectricity(%electricity) 42 electricity_from_renewables(twh) 21 electricity_from_fossil_fuels(twh) 21 access_to_electricity(%_of_population) 10 latitude 1 landarea(km2) 1 density(P/km2) 1 longitude 1 primary_energy_consumption_per_capita(kwh/person) 0 year 0 entity 0 dtype: int64
#Añadimos columna de continente al dataset para poder filtrar y hacer analisis
#Creating a column 'Continent'
def country_to_continent(country_name):
country_alpha2 = pc.country_name_to_country_alpha2(country_name)
country_continent_code = pc.country_alpha2_to_continent_code(country_alpha2)
country_continent_name = pc.convert_continent_code_to_continent_name(country_continent_code)
return country_continent_name
df['continent'] = df['entity'].apply(country_to_continent)
df.head()
entity | year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | density(P/km2) | landarea(km2) | latitude | longitude | continent | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | Afghanistan | 2000 | 1.613591 | 6.2 | 9.22 | 20000.0 | 44.99 | 0.16 | 0.0 | 0.31 | 65.957440 | 302.59482 | 1.64 | 760.000000 | NaN | NaN | NaN | 60 | 652230.0 | 33.93911 | 67.709953 | Asia |
1 | Afghanistan | 2001 | 4.074574 | 7.2 | 8.86 | 130000.0 | 45.60 | 0.09 | 0.0 | 0.50 | 84.745766 | 236.89185 | 1.74 | 730.000000 | NaN | NaN | NaN | 60 | 652230.0 | 33.93911 | 67.709953 | Asia |
2 | Afghanistan | 2002 | 9.409158 | 8.2 | 8.47 | 3950000.0 | 37.83 | 0.13 | 0.0 | 0.56 | 81.159424 | 210.86215 | 1.40 | 1029.999971 | NaN | NaN | 179.426579 | 60 | 652230.0 | 33.93911 | 67.709953 | Asia |
3 | Afghanistan | 2003 | 14.738506 | 9.5 | 8.09 | 25970000.0 | 36.66 | 0.31 | 0.0 | 0.63 | 67.021280 | 229.96822 | 1.40 | 1220.000029 | NaN | 8.832278 | 190.683814 | 60 | 652230.0 | 33.93911 | 67.709953 | Asia |
4 | Afghanistan | 2004 | 20.064968 | 10.9 | 7.75 | NaN | 44.24 | 0.33 | 0.0 | 0.56 | 62.921350 | 204.23125 | 1.20 | 1029.999971 | NaN | 1.414118 | 211.382074 | 60 | 652230.0 | 33.93911 | 67.709953 | Asia |
#Eliminamos lso dos continentes que vamos a dejar fuera del analisis y las columnas latitude y longitud
continentes_a_eliminar = ['Oceania', 'Africa']
df = df[~df['continent'].isin(continentes_a_eliminar)]
columnas_delete = ['latitude', 'longitude']
df = df.drop(columns=columnas_delete)
df.head(5)
entity | year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | density(P/km2) | landarea(km2) | continent | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | Afghanistan | 2000 | 1.613591 | 6.2 | 9.22 | 20000.0 | 44.99 | 0.16 | 0.0 | 0.31 | 65.957440 | 302.59482 | 1.64 | 760.000000 | NaN | NaN | NaN | 60 | 652230.0 | Asia |
1 | Afghanistan | 2001 | 4.074574 | 7.2 | 8.86 | 130000.0 | 45.60 | 0.09 | 0.0 | 0.50 | 84.745766 | 236.89185 | 1.74 | 730.000000 | NaN | NaN | NaN | 60 | 652230.0 | Asia |
2 | Afghanistan | 2002 | 9.409158 | 8.2 | 8.47 | 3950000.0 | 37.83 | 0.13 | 0.0 | 0.56 | 81.159424 | 210.86215 | 1.40 | 1029.999971 | NaN | NaN | 179.426579 | 60 | 652230.0 | Asia |
3 | Afghanistan | 2003 | 14.738506 | 9.5 | 8.09 | 25970000.0 | 36.66 | 0.31 | 0.0 | 0.63 | 67.021280 | 229.96822 | 1.40 | 1220.000029 | NaN | 8.832278 | 190.683814 | 60 | 652230.0 | Asia |
4 | Afghanistan | 2004 | 20.064968 | 10.9 | 7.75 | NaN | 44.24 | 0.33 | 0.0 | 0.56 | 62.921350 | 204.23125 | 1.20 | 1029.999971 | NaN | 1.414118 | 211.382074 | 60 | 652230.0 | Asia |
df.shape
(2360, 20)
df.isnull().sum().sort_values(ascending=False)
financial_flows_to_developing_countries(us$) 1527 renewables(%equivalent_primary_energy) 974 renewable_electricity_generating_capacity_per_capita 889 CO2_emissions_value_by_country_kt 303 gdp_growth 184 gdp_per_capita 180 renewable_energy_share_in_the_total_final_energy_consumption(%) 133 access_to_clean_fuels_for_cooking 127 energy_intensity_level_of_primary_energy(mj/$2017pppgdp) 118 electricity_from_nuclear(twh) 105 low_carbonelectricity(%electricity) 21 access_to_electricity(%_of_population) 1 landarea(km2) 1 density(P/km2) 1 entity 0 primary_energy_consumption_per_capita(kwh/person) 0 year 0 electricity_from_renewables(twh) 0 electricity_from_fossil_fuels(twh) 0 continent 0 dtype: int64
#Vamos a agrupar los continentes elegidos para el analisis en distintos df para seguir tratando nulos y outliers según sus características especiales
# Supongamos que ya tienes un DataFrame llamado 'df' con las columnas 'Country' y 'Continent'
# Agrupar el DataFrame por la columna 'Continent'
continent_groups = df.groupby('continent')
# Crear diccionarios para almacenar DataFrames separados por continente
continents = ['Europe', 'Asia', 'North America', 'South America',]
continent_data = {}
# Iterar sobre los grupos y almacenarlos en los diccionarios
for continent in continents:
if continent in continent_groups.groups:
continent_data[continent] = continent_groups.get_group(continent)
continent_data['Europe'].head(5)
entity | year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | density(P/km2) | landarea(km2) | continent | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
21 | Albania | 2000 | 100.0 | 38.2 | NaN | NaN | NaN | 0.14 | 0.0 | 4.55 | 97.014930 | 9029.4375 | 4.13 | 3170.000000 | NaN | 6.946217 | 1126.683340 | 105 | 28748.0 | Europe |
22 | Albania | 2001 | 100.0 | 40.5 | NaN | NaN | NaN | 0.13 | 0.0 | 3.52 | 96.438354 | 8635.5320 | 3.89 | 3230.000000 | NaN | 8.293313 | 1281.659826 | 105 | 28748.0 | Europe |
23 | Albania | 2002 | 100.0 | 43.2 | NaN | NaN | NaN | 0.16 | 0.0 | 3.48 | 95.604390 | 9443.5550 | 4.10 | 3759.999990 | NaN | 4.536524 | 1425.124219 | 105 | 28748.0 | Europe |
24 | Albania | 2003 | 100.0 | 46.4 | NaN | NaN | NaN | 0.10 | 0.0 | 5.12 | 98.084300 | 10756.6120 | 3.80 | 4070.000172 | NaN | 5.528637 | 1846.120121 | 105 | 28748.0 | Europe |
25 | Albania | 2004 | 100.0 | 49.0 | NaN | NaN | NaN | 0.13 | 0.0 | 5.41 | 97.653430 | 11586.9510 | 3.96 | 4250.000000 | NaN | 5.514668 | 2373.581292 | 105 | 28748.0 | Europe |
continent_data['Europe'].isnull().sum().sort_values(ascending=False)
renewable_electricity_generating_capacity_per_capita 763 financial_flows_to_developing_countries(us$) 763 renewables(%equivalent_primary_energy) 91 renewable_energy_share_in_the_total_final_energy_consumption(%) 57 CO2_emissions_value_by_country_kt 57 gdp_per_capita 42 gdp_growth 42 energy_intensity_level_of_primary_energy(mj/$2017pppgdp) 37 access_to_clean_fuels_for_cooking 21 entity 0 landarea(km2) 0 density(P/km2) 0 low_carbonelectricity(%electricity) 0 primary_energy_consumption_per_capita(kwh/person) 0 year 0 electricity_from_renewables(twh) 0 electricity_from_nuclear(twh) 0 electricity_from_fossil_fuels(twh) 0 access_to_electricity(%_of_population) 0 continent 0 dtype: int64
#Rellenamos columnas de nulos con 0
continent_data['Europe']['financial_flows_to_developing_countries(us$)'].fillna(0, inplace=True)
continent_data['Europe']['renewables(%equivalent_primary_energy)'].fillna(0, inplace=True)
continent_data['Europe']['renewable_electricity_generating_capacity_per_capita'].fillna(0, inplace=True)
continent_data['Europe']['CO2_emissions_value_by_country_kt'].fillna(0, inplace=True)
C:\Users\danie\AppData\Local\Temp\ipykernel_17188\2168471866.py:2: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy continent_data['Europe']['financial_flows_to_developing_countries(us$)'].fillna(0, inplace=True) C:\Users\danie\AppData\Local\Temp\ipykernel_17188\2168471866.py:3: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy continent_data['Europe']['renewables(%equivalent_primary_energy)'].fillna(0, inplace=True) C:\Users\danie\AppData\Local\Temp\ipykernel_17188\2168471866.py:4: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy continent_data['Europe']['renewable_electricity_generating_capacity_per_capita'].fillna(0, inplace=True) C:\Users\danie\AppData\Local\Temp\ipykernel_17188\2168471866.py:5: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy continent_data['Europe']['CO2_emissions_value_by_country_kt'].fillna(0, inplace=True)
continent_data['Europe'] = continent_data['Europe'].interpolate(method='pad')
continent_data['Europe']['renewable_energy_share_in_the_total_final_energy_consumption(%)'].fillna(0, inplace=True)
#Save numerical data of europe
df_num_europe= continent_data['Europe'].select_dtypes(exclude=["object"])
df_num_europe.head(10)
year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | landarea(km2) | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
21 | 2000 | 100.0 | 38.2 | 0.0 | 0.0 | 0.0 | 0.14 | 0.0 | 4.55 | 97.014930 | 9029.4375 | 4.13 | 3170.000000 | 0.0 | 6.946217 | 1126.683340 | 28748.0 |
22 | 2001 | 100.0 | 40.5 | 0.0 | 0.0 | 0.0 | 0.13 | 0.0 | 3.52 | 96.438354 | 8635.5320 | 3.89 | 3230.000000 | 0.0 | 8.293313 | 1281.659826 | 28748.0 |
23 | 2002 | 100.0 | 43.2 | 0.0 | 0.0 | 0.0 | 0.16 | 0.0 | 3.48 | 95.604390 | 9443.5550 | 4.10 | 3759.999990 | 0.0 | 4.536524 | 1425.124219 | 28748.0 |
24 | 2003 | 100.0 | 46.4 | 0.0 | 0.0 | 0.0 | 0.10 | 0.0 | 5.12 | 98.084300 | 10756.6120 | 3.80 | 4070.000172 | 0.0 | 5.528637 | 1846.120121 | 28748.0 |
25 | 2004 | 100.0 | 49.0 | 0.0 | 0.0 | 0.0 | 0.13 | 0.0 | 5.41 | 97.653430 | 11586.9510 | 3.96 | 4250.000000 | 0.0 | 5.514668 | 2373.581292 | 28748.0 |
26 | 2005 | 100.0 | 51.9 | 0.0 | 0.0 | 0.0 | 0.07 | 0.0 | 5.32 | 98.701300 | 11545.6160 | 3.75 | 4030.000210 | 0.0 | 5.526424 | 2673.786584 | 28748.0 |
27 | 2006 | 100.0 | 54.8 | 0.0 | 0.0 | 0.0 | 0.09 | 0.0 | 4.95 | 98.214280 | 10976.7280 | 3.46 | 4010.000229 | 0.0 | 5.902659 | 2972.742924 | 28748.0 |
28 | 2007 | 100.0 | 58.1 | 0.0 | 0.0 | 0.0 | 0.07 | 0.0 | 2.76 | 97.526505 | 9534.8740 | 3.12 | 4139.999866 | 0.0 | 5.983260 | 3595.038057 | 28748.0 |
29 | 2008 | 100.0 | 60.8 | 0.0 | 0.0 | 0.0 | 0.07 | 0.0 | 3.76 | 98.172325 | 10499.3280 | 3.01 | 4079.999924 | 0.0 | 7.500041 | 4370.539925 | 28748.0 |
30 | 2009 | 100.0 | 63.6 | 0.0 | 0.0 | 0.0 | 0.00 | 0.0 | 5.20 | 100.000000 | 11524.4280 | 2.98 | 4219.999790 | 0.0 | 3.354289 | 4114.134899 | 28748.0 |
df_num_europe = df_num_europe.reset_index(drop=True)
df_num_europe.isnull().sum().sort_values(ascending=False)
year 0 low_carbonelectricity(%electricity) 0 gdp_per_capita 0 gdp_growth 0 renewables(%equivalent_primary_energy) 0 CO2_emissions_value_by_country_kt 0 energy_intensity_level_of_primary_energy(mj/$2017pppgdp) 0 primary_energy_consumption_per_capita(kwh/person) 0 electricity_from_renewables(twh) 0 access_to_electricity(%_of_population) 0 electricity_from_nuclear(twh) 0 electricity_from_fossil_fuels(twh) 0 renewable_energy_share_in_the_total_final_energy_consumption(%) 0 financial_flows_to_developing_countries(us$) 0 renewable_electricity_generating_capacity_per_capita 0 access_to_clean_fuels_for_cooking 0 landarea(km2) 0 dtype: int64
#CHECKING OUTLIERS
for col in df_num_europe.columns:
# Crea un nuevo subplot para cada gráfico
fig, ax = plt.subplots(figsize=(8, 6))
# Crea el gráfico de boxplot utilizando seaborn
sns.boxplot(x=df[col], ax=ax)
# Establece el título de cada gráfico
ax.set_title('Boxplot Europe by {}'.format(col))
# Muestra cada gráfico
plt.show()
df_num_europe['access_to_electricity(%_of_population)'].describe()
count 763.000000 mean 99.981129 std 0.133596 min 97.700000 25% 100.000000 50% 100.000000 75% 100.000000 max 100.000000 Name: access_to_electricity(%_of_population), dtype: float64
columns_with_outliers_europe = []
for column in df_num_europe.columns: if df_num_europe[column].dtype == 'float64' or df_num_europe[column].dtype == 'int64': Q1 = df_num_europe[column].quantile(0.25) Q3 = df_num_europe[column].quantile(0.75) IQR = Q3 - Q1
lower_limit = Q1 - 1.5 * IQR
upper_limit = Q3 + 1.5 * IQR
outliers = df_num_europe[
(df_num_europe[column] > upper_limit) |
(df_num_europe[column] < lower_limit)
]
if not outliers.empty:
print(f"Columna: {column}")
print(outliers)
columns_with_outliers_europe.append(column)
print("Columnas con outliers y número de outliers:") for column, num_outliers in columns_with_outliers_europe.items(): print(f"{column}: {num_outliers}")
#Totol number of outliers by column
column_outlier_total_eu = {}
# Calcular el IQR para cada columna numérica y encontrar outliers
for column in df_num_europe.columns:
if df_num_europe[column].dtype == 'float64' or df_num_europe[column].dtype == 'int64':
Q1 = df_num_europe[column].quantile(0.25)
Q3 = df_num_europe[column].quantile(0.75)
IQR = Q3 - Q1
lower_limit = Q1 - 1.5 * IQR
upper_limit = Q3 + 1.5 * IQR
outliers = df_num_europe[
(df_num_europe[column] > upper_limit) |
(df_num_europe[column] < lower_limit)
]
num_outliers = len(outliers)
column_outlier_total_eu[column] = num_outliers
# Mostrar el número total de outliers por columna
for column, num_outliers in column_outlier_total_eu.items():
print(f"Número de outliers en {column}: {num_outliers}")
Número de outliers en year: 0 Número de outliers en access_to_electricity(%_of_population): 31 Número de outliers en access_to_clean_fuels_for_cooking: 175 Número de outliers en renewable_electricity_generating_capacity_per_capita: 0 Número de outliers en financial_flows_to_developing_countries(us$): 0 Número de outliers en renewable_energy_share_in_the_total_final_energy_consumption(%): 44 Número de outliers en electricity_from_fossil_fuels(twh): 99 Número de outliers en electricity_from_nuclear(twh): 111 Número de outliers en electricity_from_renewables(twh): 100 Número de outliers en low_carbonelectricity(%electricity): 0 Número de outliers en primary_energy_consumption_per_capita(kwh/person): 54 Número de outliers en energy_intensity_level_of_primary_energy(mj/$2017pppgdp): 45 Número de outliers en CO2_emissions_value_by_country_kt: 135 Número de outliers en renewables(%equivalent_primary_energy): 47 Número de outliers en gdp_growth: 76 Número de outliers en gdp_per_capita: 17 Número de outliers en landarea(km2): 42
df_num_europe.shape
(763, 17)
#CHECKING DISTRIBUTION
for c in df_num_europe.columns:
plt.figure(figsize=(10,4))
sns.histplot(df_num_europe[c], kde= True)
plt.title(c)
plt.show()
continent_data['Asia'].head()
entity | year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | density(P/km2) | landarea(km2) | continent | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | Afghanistan | 2000 | 1.613591 | 6.2 | 9.22 | 20000.0 | 44.99 | 0.16 | 0.0 | 0.31 | 65.957440 | 302.59482 | 1.64 | 760.000000 | NaN | NaN | NaN | 60 | 652230.0 | Asia |
1 | Afghanistan | 2001 | 4.074574 | 7.2 | 8.86 | 130000.0 | 45.60 | 0.09 | 0.0 | 0.50 | 84.745766 | 236.89185 | 1.74 | 730.000000 | NaN | NaN | NaN | 60 | 652230.0 | Asia |
2 | Afghanistan | 2002 | 9.409158 | 8.2 | 8.47 | 3950000.0 | 37.83 | 0.13 | 0.0 | 0.56 | 81.159424 | 210.86215 | 1.40 | 1029.999971 | NaN | NaN | 179.426579 | 60 | 652230.0 | Asia |
3 | Afghanistan | 2003 | 14.738506 | 9.5 | 8.09 | 25970000.0 | 36.66 | 0.31 | 0.0 | 0.63 | 67.021280 | 229.96822 | 1.40 | 1220.000029 | NaN | 8.832278 | 190.683814 | 60 | 652230.0 | Asia |
4 | Afghanistan | 2004 | 20.064968 | 10.9 | 7.75 | NaN | 44.24 | 0.33 | 0.0 | 0.56 | 62.921350 | 204.23125 | 1.20 | 1029.999971 | NaN | 1.414118 | 211.382074 | 60 | 652230.0 | Asia |
continent_data['Asia'].shape
(819, 20)
continent_data['Asia'].isnull().sum().sort_values(ascending=False)
financial_flows_to_developing_countries(us$) 356 renewables(%equivalent_primary_energy) 315 CO2_emissions_value_by_country_kt 99 electricity_from_nuclear(twh) 84 renewable_electricity_generating_capacity_per_capita 63 gdp_growth 47 gdp_per_capita 45 energy_intensity_level_of_primary_energy(mj/$2017pppgdp) 43 renewable_energy_share_in_the_total_final_energy_consumption(%) 39 access_to_clean_fuels_for_cooking 21 entity 0 landarea(km2) 0 density(P/km2) 0 low_carbonelectricity(%electricity) 0 primary_energy_consumption_per_capita(kwh/person) 0 year 0 electricity_from_renewables(twh) 0 electricity_from_fossil_fuels(twh) 0 access_to_electricity(%_of_population) 0 continent 0 dtype: int64
continent_data['Asia'] = continent_data['Asia'].interpolate(method='pad')
continent_data['Asia'].isnull().sum().sort_values(ascending=False)
entity 0 year 0 landarea(km2) 0 density(P/km2) 0 gdp_per_capita 0 gdp_growth 0 renewables(%equivalent_primary_energy) 0 CO2_emissions_value_by_country_kt 0 energy_intensity_level_of_primary_energy(mj/$2017pppgdp) 0 primary_energy_consumption_per_capita(kwh/person) 0 low_carbonelectricity(%electricity) 0 electricity_from_renewables(twh) 0 electricity_from_nuclear(twh) 0 electricity_from_fossil_fuels(twh) 0 renewable_energy_share_in_the_total_final_energy_consumption(%) 0 financial_flows_to_developing_countries(us$) 0 renewable_electricity_generating_capacity_per_capita 0 access_to_clean_fuels_for_cooking 0 access_to_electricity(%_of_population) 0 continent 0 dtype: int64
continent_data['Asia']['gdp_per_capita'].fillna(0, inplace=True)
continent_data['Asia']['gdp_growth'].fillna(0, inplace=True)
continent_data['Asia']['renewables(%equivalent_primary_energy)'].fillna(0, inplace=True)
#Save numerical data of Asia
df_num_asia= continent_data['Asia'].select_dtypes(exclude=["object"])
df_num_asia.head()
year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | landarea(km2) | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | 2000 | 1.613591 | 6.2 | 9.22 | 20000.0 | 44.99 | 0.16 | 0.0 | 0.31 | 65.957440 | 302.59482 | 1.64 | 760.000000 | 0.0 | 0.000000 | 0.000000 | 652230.0 |
1 | 2001 | 4.074574 | 7.2 | 8.86 | 130000.0 | 45.60 | 0.09 | 0.0 | 0.50 | 84.745766 | 236.89185 | 1.74 | 730.000000 | 0.0 | 0.000000 | 0.000000 | 652230.0 |
2 | 2002 | 9.409158 | 8.2 | 8.47 | 3950000.0 | 37.83 | 0.13 | 0.0 | 0.56 | 81.159424 | 210.86215 | 1.40 | 1029.999971 | 0.0 | 0.000000 | 179.426579 | 652230.0 |
3 | 2003 | 14.738506 | 9.5 | 8.09 | 25970000.0 | 36.66 | 0.31 | 0.0 | 0.63 | 67.021280 | 229.96822 | 1.40 | 1220.000029 | 0.0 | 8.832278 | 190.683814 | 652230.0 |
4 | 2004 | 20.064968 | 10.9 | 7.75 | 25970000.0 | 44.24 | 0.33 | 0.0 | 0.56 | 62.921350 | 204.23125 | 1.20 | 1029.999971 | 0.0 | 1.414118 | 211.382074 | 652230.0 |
df_num_asia = df_num_asia.reset_index(drop=True)
#CHECKING OUTLIERS
for col in df_num_asia.columns:
# Crea un nuevo subplot para cada gráfico
fig, ax = plt.subplots(figsize=(8, 6))
# Crea el gráfico de boxplot utilizando seaborn
sns.boxplot(x=df[col], ax=ax)
# Establece el título de cada gráfico
ax.set_title('Boxplot Asia by {}'.format(col))
# Muestra cada gráfico
plt.show()
df_num_asia.describe()
year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | landarea(km2) | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
count | 819.000000 | 819.000000 | 819.000000 | 819.000000 | 8.190000e+02 | 819.000000 | 819.000000 | 819.000000 | 819.000000 | 819.000000 | 819.00000 | 819.000000 | 8.190000e+02 | 819.000000 | 819.000000 | 819.000000 | 8.190000e+02 |
mean | 2010.000000 | 89.540016 | 70.617277 | 167.388669 | 1.099556e+08 | 20.984762 | 161.661331 | 9.334151 | 38.174103 | 25.889534 | 36163.68892 | 5.803985 | 3.540682e+05 | 4.477346 | 4.630280 | 11352.759007 | 7.493200e+05 |
std | 6.059001 | 18.919985 | 31.708110 | 346.710720 | 2.395457e+08 | 26.365989 | 571.158550 | 41.483333 | 179.314413 | 32.790328 | 50354.61042 | 4.048257 | 1.313613e+06 | 4.951117 | 5.880122 | 15502.219185 | 1.626458e+06 |
min | 2000.000000 | 1.613591 | 2.900000 | 0.000000 | 0.000000e+00 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 204.23125 | 1.200000 | 2.100000e+02 | 0.000000 | -36.658153 | 0.000000 | 2.980000e+02 |
25% | 2005.000000 | 87.550000 | 42.450000 | 7.095000 | 8.100000e+05 | 1.115000 | 3.175000 | 0.000000 | 0.040000 | 0.109167 | 5777.91030 | 3.445000 | 1.240500e+04 | 0.040307 | 2.175678 | 1286.232553 | 3.839400e+04 |
50% | 2010.000000 | 99.604300 | 82.900000 | 55.080000 | 1.117000e+07 | 6.490000 | 21.950000 | 0.000000 | 2.240000 | 10.663012 | 16075.37400 | 4.820000 | 6.055000e+04 | 2.907140 | 5.007683 | 4021.743306 | 1.999510e+05 |
75% | 2015.000000 | 100.000000 | 100.000000 | 120.570000 | 1.200950e+08 | 34.405000 | 73.460000 | 0.000000 | 11.510000 | 37.677067 | 36441.39650 | 6.660000 | 1.502500e+05 | 7.163172 | 7.298275 | 16551.940215 | 6.522300e+05 |
max | 2020.000000 | 100.000000 | 100.000000 | 3060.190000 | 3.386850e+09 | 93.460000 | 5184.130000 | 366.200000 | 2184.940000 | 100.000010 | 262585.70000 | 32.570000 | 1.070722e+07 | 24.803470 | 53.381794 | 85075.986540 | 9.596960e+06 |
columns_with_outliers_asia = []
for column in df_num_asia.columns: if df_num_asia[column].dtype == 'float64' or df_num_asia[column].dtype == 'int64': Q1 = df_num_asia[column].quantile(0.25) Q3 = df_num_asia[column].quantile(0.75) IQR = Q3 - Q1
lower_limit = Q1 - 1.5 * IQR
upper_limit = Q3 + 1.5 * IQR
outliers = df_num_asia[
(df_num_asia[column] > upper_limit) |
(df_num_asia[column] < lower_limit)
]
if not outliers.empty:
print(f"Columna: {column}")
print(outliers)
columns_with_outliers_asia.append(column)
print("Columnas con outliers y número de outliers:") for column, num_outliers in columns_with_outliers_asia.items(): print(f"{column}: {num_outliers}")
#Totol number of outliers by column Asia
column_outlier_total_asia = {}
# Calcular el IQR para cada columna numérica y encontrar outliers
for column in df_num_asia.columns:
if df_num_asia[column].dtype == 'float64' or df_num_asia[column].dtype == 'int64':
Q1 = df_num_asia[column].quantile(0.25)
Q3 = df_num_asia[column].quantile(0.75)
IQR = Q3 - Q1
lower_limit = Q1 - 1.5 * IQR
upper_limit = Q3 + 1.5 * IQR
outliers = df_num_asia[
(df_num_asia[column] > upper_limit) |
(df_num_asia[column] < lower_limit)
]
num_outliers = len(outliers)
column_outlier_total_asia[column] = num_outliers
# Mostrar el número total de outliers por columna
for column, num_outliers in column_outlier_total_asia.items():
print(f"Número de outliers en {column}: {num_outliers}")
Número de outliers en year: 0 Número de outliers en access_to_electricity(%_of_population): 110 Número de outliers en access_to_clean_fuels_for_cooking: 0 Número de outliers en renewable_electricity_generating_capacity_per_capita: 142 Número de outliers en financial_flows_to_developing_countries(us$): 92 Número de outliers en renewable_energy_share_in_the_total_final_energy_consumption(%): 35 Número de outliers en electricity_from_fossil_fuels(twh): 91 Número de outliers en electricity_from_nuclear(twh): 126 Número de outliers en electricity_from_renewables(twh): 111 Número de outliers en low_carbonelectricity(%electricity): 60 Número de outliers en primary_energy_consumption_per_capita(kwh/person): 116 Número de outliers en energy_intensity_level_of_primary_energy(mj/$2017pppgdp): 44 Número de outliers en CO2_emissions_value_by_country_kt: 91 Número de outliers en renewables(%equivalent_primary_energy): 10 Número de outliers en gdp_growth: 59 Número de outliers en gdp_per_capita: 58 Número de outliers en landarea(km2): 105
df_num_asia.shape
(819, 17)
#CHECKING DISTRIBUTION
for c in df_num_asia.columns:
plt.figure(figsize=(10,4))
sns.histplot(df_num_asia[c], kde= True)
plt.title(c)
plt.show()
continent_data['North America'].head(5)
entity | year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | density(P/km2) | landarea(km2) | continent | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
84 | Antigua and Barbuda | 2000 | 97.689260 | 100.0 | 0.0 | NaN | 0.0 | 0.14 | 0.0 | 0.0 | 0.0 | 28457.223 | 3.13 | 330.000000 | NaN | 6.203431 | 10872.29295 | 223 | 443.0 | North America |
85 | Antigua and Barbuda | 2001 | 97.785255 | 100.0 | 0.0 | NaN | 0.0 | 0.16 | 0.0 | 0.0 | 0.0 | 28025.290 | 3.31 | 350.000000 | NaN | -4.548003 | 10367.31961 | 223 | 443.0 | North America |
86 | Antigua and Barbuda | 2002 | 100.000000 | 100.0 | 0.0 | NaN | 0.0 | 0.18 | 0.0 | 0.0 | 0.0 | 29660.344 | 3.59 | 389.999986 | NaN | 1.027442 | 10401.05088 | 223 | 443.0 | North America |
87 | Antigua and Barbuda | 2003 | 97.956825 | 100.0 | 0.0 | NaN | 0.0 | 0.20 | 0.0 | 0.0 | 0.0 | 30824.252 | 3.63 | 409.999996 | NaN | 6.076544 | 10797.95106 | 223 | 443.0 | North America |
88 | Antigua and Barbuda | 2004 | 98.037100 | 100.0 | 0.0 | NaN | 0.0 | 0.21 | 0.0 | 0.0 | 0.0 | 31622.059 | 3.57 | 400.000006 | NaN | 5.767029 | 11446.96914 | 223 | 443.0 | North America |
continent_data['North America'].shape
(567, 20)
continent_data['North America'].isnull().sum().sort_values(ascending=False)
renewables(%equivalent_primary_energy) 483 financial_flows_to_developing_countries(us$) 348 CO2_emissions_value_by_country_kt 136 gdp_growth 94 gdp_per_capita 92 access_to_clean_fuels_for_cooking 84 renewable_electricity_generating_capacity_per_capita 63 energy_intensity_level_of_primary_energy(mj/$2017pppgdp) 27 renewable_energy_share_in_the_total_final_energy_consumption(%) 27 low_carbonelectricity(%electricity) 21 landarea(km2) 0 density(P/km2) 0 entity 0 primary_energy_consumption_per_capita(kwh/person) 0 year 0 electricity_from_renewables(twh) 0 electricity_from_nuclear(twh) 0 electricity_from_fossil_fuels(twh) 0 access_to_electricity(%_of_population) 0 continent 0 dtype: int64
continent_data['North America'] = continent_data['North America'].interpolate(method='pad')
continent_data['North America']['renewables(%equivalent_primary_energy)'].fillna(0, inplace=True)
continent_data['North America']['financial_flows_to_developing_countries(us$)'].fillna(0, inplace=True)
continent_data['North America'].isnull().sum().sort_values(ascending=False)
entity 0 year 0 landarea(km2) 0 density(P/km2) 0 gdp_per_capita 0 gdp_growth 0 renewables(%equivalent_primary_energy) 0 CO2_emissions_value_by_country_kt 0 energy_intensity_level_of_primary_energy(mj/$2017pppgdp) 0 primary_energy_consumption_per_capita(kwh/person) 0 low_carbonelectricity(%electricity) 0 electricity_from_renewables(twh) 0 electricity_from_nuclear(twh) 0 electricity_from_fossil_fuels(twh) 0 renewable_energy_share_in_the_total_final_energy_consumption(%) 0 financial_flows_to_developing_countries(us$) 0 renewable_electricity_generating_capacity_per_capita 0 access_to_clean_fuels_for_cooking 0 access_to_electricity(%_of_population) 0 continent 0 dtype: int64
#Save numerical data of Asia
df_num_north_america= continent_data['North America'].select_dtypes(exclude=["object"])
df_num_north_america.head()
year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | landarea(km2) | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
84 | 2000 | 97.689260 | 100.0 | 0.0 | 0.0 | 0.0 | 0.14 | 0.0 | 0.0 | 0.0 | 28457.223 | 3.13 | 330.000000 | 0.0 | 6.203431 | 10872.29295 | 443.0 |
85 | 2001 | 97.785255 | 100.0 | 0.0 | 0.0 | 0.0 | 0.16 | 0.0 | 0.0 | 0.0 | 28025.290 | 3.31 | 350.000000 | 0.0 | -4.548003 | 10367.31961 | 443.0 |
86 | 2002 | 100.000000 | 100.0 | 0.0 | 0.0 | 0.0 | 0.18 | 0.0 | 0.0 | 0.0 | 29660.344 | 3.59 | 389.999986 | 0.0 | 1.027442 | 10401.05088 | 443.0 |
87 | 2003 | 97.956825 | 100.0 | 0.0 | 0.0 | 0.0 | 0.20 | 0.0 | 0.0 | 0.0 | 30824.252 | 3.63 | 409.999996 | 0.0 | 6.076544 | 10797.95106 | 443.0 |
88 | 2004 | 98.037100 | 100.0 | 0.0 | 0.0 | 0.0 | 0.21 | 0.0 | 0.0 | 0.0 | 31622.059 | 3.57 | 400.000006 | 0.0 | 5.767029 | 11446.96914 | 443.0 |
df_num_north_america = df_num_north_america.reset_index(drop=True)
#CHECKING OUTLIERS
for col in df_num_north_america.columns:
# Crea un nuevo subplot para cada gráfico
fig, ax = plt.subplots(figsize=(8, 6))
# Crea el gráfico de boxplot utilizando seaborn
sns.boxplot(x=df[col], ax=ax)
# Establece el título de cada gráfico
ax.set_title('Boxplot North America by {}'.format(col))
# Muestra cada gráfico
plt.show()
df_num_north_america.describe()
year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | landarea(km2) | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
count | 567.000000 | 567.000000 | 567.000000 | 567.000000 | 5.670000e+02 | 567.000000 | 567.000000 | 567.000000 | 567.000000 | 567.000000 | 567.000000 | 567.000000 | 5.670000e+02 | 567.000000 | 567.000000 | 567.000000 | 5.670000e+02 |
mean | 2010.000000 | 92.509613 | 83.251411 | 103.868571 | 2.622497e+07 | 19.401570 | 117.229630 | 32.896931 | 35.443245 | 26.875820 | 29161.290014 | 4.359506 | 2.402522e+05 | 16.066678 | 1.257890 | 20246.812215 | 8.346131e+05 |
std | 6.060648 | 12.981611 | 23.374437 | 126.238442 | 6.502837e+07 | 21.131713 | 518.454273 | 149.695622 | 119.208969 | 28.162265 | 33608.641993 | 3.906855 | 1.004951e+06 | 13.247613 | 4.440433 | 22751.669692 | 2.595081e+06 |
min | 2000.000000 | 31.542381 | 2.600000 | 0.000000 | 0.000000e+00 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 836.080400 | 0.110000 | 1.200000e+02 | 0.000000 | -20.192371 | 542.333596 | 5.400000e+01 |
25% | 2005.000000 | 89.845680 | 82.450000 | 4.280000 | 2.600000e+05 | 1.770000 | 0.265000 | 0.000000 | 0.000000 | 0.114616 | 9050.244500 | 2.440000 | 8.837470e+02 | 0.042004 | -1.331781 | 4404.510041 | 4.300000e+02 |
50% | 2010.000000 | 98.037100 | 89.600000 | 61.700000 | 3.100000e+06 | 10.570000 | 1.630000 | 0.000000 | 0.170000 | 16.666668 | 15259.877000 | 3.330000 | 7.560000e+03 | 9.475372 | 1.999999 | 10744.091020 | 1.387800e+04 |
75% | 2015.000000 | 100.000000 | 100.000000 | 138.850000 | 1.955000e+07 | 29.090000 | 6.515000 | 0.000000 | 3.435000 | 48.692244 | 35855.724500 | 4.860000 | 1.334000e+04 | 30.542150 | 3.842537 | 32290.921140 | 1.088890e+05 |
max | 2020.000000 | 100.000000 | 100.000000 | 622.840000 | 5.963700e+08 | 82.190000 | 2988.240000 | 809.410000 | 821.400000 | 99.121710 | 165831.280000 | 21.600000 | 5.775810e+06 | 30.542150 | 14.440990 | 117098.446300 | 9.984670e+06 |
columns_with_outliers_north_america = []
for column in df_num_north_america.columns: if df_num_north_america[column].dtype == 'float64' or df_num_north_america[column].dtype == 'int64': Q1 = df_num_north_america[column].quantile(0.25) Q3 = df_num_north_america[column].quantile(0.75) IQR = Q3 - Q1
lower_limit = Q1 - 1.5 * IQR
upper_limit = Q3 + 1.5 * IQR
outliers = df_num_north_america[
(df_num_north_america[column] > upper_limit) |
(df_num_north_america[column] < lower_limit)
]
if not outliers.empty:
print(f"Columna: {column}")
print(outliers)
columns_with_outliers_north_america.append(column)
print("Columnas con outliers y número de outliers:") for column, num_outliers in columns_with_outliers_north_america.items(): print(f"{column}: {num_outliers}")
#Total number of outliers by column
column_outlier_total_north_america = {}
# Calcular el IQR para cada columna numérica y encontrar outliers
for column in df_num_north_america.columns:
if df_num_north_america[column].dtype == 'float64' or df_num_north_america[column].dtype == 'int64':
Q1 = df_num_north_america[column].quantile(0.25)
Q3 = df_num_north_america[column].quantile(0.75)
IQR = Q3 - Q1
lower_limit = Q1 - 1.5 * IQR
upper_limit = Q3 + 1.5 * IQR
outliers = df_num_north_america[
(df_num_north_america[column] > upper_limit) |
(df_num_north_america[column] < lower_limit)
]
num_outliers = len(outliers)
column_outlier_total_north_america[column] = num_outliers
# Mostrar el número total de outliers por columna
for column, num_outliers in column_outlier_total_north_america.items():
print(f"Número de outliers en {column}: {num_outliers}")
Número de outliers en year: 0 Número de outliers en access_to_electricity(%_of_population): 36 Número de outliers en access_to_clean_fuels_for_cooking: 83 Número de outliers en renewable_electricity_generating_capacity_per_capita: 38 Número de outliers en financial_flows_to_developing_countries(us$): 79 Número de outliers en renewable_energy_share_in_the_total_final_energy_consumption(%): 21 Número de outliers en electricity_from_fossil_fuels(twh): 95 Número de outliers en electricity_from_nuclear(twh): 63 Número de outliers en electricity_from_renewables(twh): 80 Número de outliers en low_carbonelectricity(%electricity): 0 Número de outliers en primary_energy_consumption_per_capita(kwh/person): 62 Número de outliers en energy_intensity_level_of_primary_energy(mj/$2017pppgdp): 38 Número de outliers en CO2_emissions_value_by_country_kt: 63 Número de outliers en renewables(%equivalent_primary_energy): 0 Número de outliers en gdp_growth: 17 Número de outliers en gdp_per_capita: 27 Número de outliers en landarea(km2): 63
df_num_north_america.shape
(567, 17)
#CHECKING DISTRIBUTION
for c in df_num_north_america.columns:
plt.figure(figsize=(10,4))
sns.histplot(df_num_north_america[c], kde= True)
plt.title(c)
plt.show()
continent_data['South America'].head(5)
entity | year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | density(P/km2) | landarea(km2) | continent | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
105 | Argentina | 2000 | 95.783290 | 95.00 | 235.62 | NaN | 9.85 | 50.37 | 5.99 | 28.89 | 40.914955 | 18849.810 | 3.69 | 132270.0000 | 12.283858 | -0.788999 | 7708.099115 | 17 | 2780400.0 | South America |
106 | Argentina | 2001 | 95.511060 | 95.60 | 234.09 | 3890000.0 | 11.51 | 42.91 | 6.54 | 37.04 | 50.387330 | 18210.120 | 3.59 | 125260.0000 | 15.937488 | -4.408840 | 7208.373114 | 17 | 2780400.0 | South America |
107 | Argentina | 2002 | 96.228870 | 96.10 | 236.78 | NaN | 11.62 | 39.70 | 5.39 | 36.06 | 51.078250 | 17275.256 | 3.99 | 117470.0012 | 16.249306 | -10.894485 | 2593.404563 | 17 | 2780400.0 | South America |
108 | Argentina | 2003 | 96.442635 | 96.45 | 234.82 | 50000.0 | 10.82 | 46.73 | 7.03 | 34.38 | 46.982075 | 18326.521 | 3.97 | 127660.0037 | 14.408259 | 8.837041 | 3349.806300 | 17 | 2780400.0 | South America |
109 | Argentina | 2004 | 96.653500 | 97.20 | 236.15 | 40960000.0 | 9.29 | 57.35 | 7.31 | 31.03 | 40.066883 | 19261.457 | 4.04 | 141380.0049 | 12.185576 | 9.029573 | 4277.721573 | 17 | 2780400.0 | South America |
continent_data['South America'].shape
(211, 20)
continent_data['South America'].isnull().sum().sort_values(ascending=False)
renewables(%equivalent_primary_energy) 85 financial_flows_to_developing_countries(us$) 60 electricity_from_nuclear(twh) 21 CO2_emissions_value_by_country_kt 11 energy_intensity_level_of_primary_energy(mj/$2017pppgdp) 11 renewable_energy_share_in_the_total_final_energy_consumption(%) 10 access_to_electricity(%_of_population) 1 access_to_clean_fuels_for_cooking 1 landarea(km2) 1 density(P/km2) 1 gdp_per_capita 1 gdp_growth 1 entity 0 low_carbonelectricity(%electricity) 0 primary_energy_consumption_per_capita(kwh/person) 0 year 0 electricity_from_renewables(twh) 0 electricity_from_fossil_fuels(twh) 0 renewable_electricity_generating_capacity_per_capita 0 continent 0 dtype: int64
continent_data['South America'] = continent_data['South America'].interpolate(method='pad')
continent_data['South America'].isnull().sum().sort_values(ascending=False)
financial_flows_to_developing_countries(us$) 1 entity 0 primary_energy_consumption_per_capita(kwh/person) 0 landarea(km2) 0 density(P/km2) 0 gdp_per_capita 0 gdp_growth 0 renewables(%equivalent_primary_energy) 0 CO2_emissions_value_by_country_kt 0 energy_intensity_level_of_primary_energy(mj/$2017pppgdp) 0 low_carbonelectricity(%electricity) 0 year 0 electricity_from_renewables(twh) 0 electricity_from_nuclear(twh) 0 electricity_from_fossil_fuels(twh) 0 renewable_energy_share_in_the_total_final_energy_consumption(%) 0 renewable_electricity_generating_capacity_per_capita 0 access_to_clean_fuels_for_cooking 0 access_to_electricity(%_of_population) 0 continent 0 dtype: int64
continent_data['South America']['financial_flows_to_developing_countries(us$)'].fillna(0, inplace=True)
#Save numerical data of Asia
df_num_south_america= continent_data['South America'].select_dtypes(exclude=["object"])
df_num_south_america.head(5)
year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | landarea(km2) | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
105 | 2000 | 95.783290 | 95.00 | 235.62 | 0.0 | 9.85 | 50.37 | 5.99 | 28.89 | 40.914955 | 18849.810 | 3.69 | 132270.0000 | 12.283858 | -0.788999 | 7708.099115 | 2780400.0 |
106 | 2001 | 95.511060 | 95.60 | 234.09 | 3890000.0 | 11.51 | 42.91 | 6.54 | 37.04 | 50.387330 | 18210.120 | 3.59 | 125260.0000 | 15.937488 | -4.408840 | 7208.373114 | 2780400.0 |
107 | 2002 | 96.228870 | 96.10 | 236.78 | 3890000.0 | 11.62 | 39.70 | 5.39 | 36.06 | 51.078250 | 17275.256 | 3.99 | 117470.0012 | 16.249306 | -10.894485 | 2593.404563 | 2780400.0 |
108 | 2003 | 96.442635 | 96.45 | 234.82 | 50000.0 | 10.82 | 46.73 | 7.03 | 34.38 | 46.982075 | 18326.521 | 3.97 | 127660.0037 | 14.408259 | 8.837041 | 3349.806300 | 2780400.0 |
109 | 2004 | 96.653500 | 97.20 | 236.15 | 40960000.0 | 9.29 | 57.35 | 7.31 | 31.03 | 40.066883 | 19261.457 | 4.04 | 141380.0049 | 12.185576 | 9.029573 | 4277.721573 | 2780400.0 |
df_num_south_america = df_num_south_america.reset_index(drop=True)
#CHECKING OUTLIERS
for col in df_num_south_america.columns:
# Crea un nuevo subplot para cada gráfico
fig, ax = plt.subplots(figsize=(8, 6))
# Crea el gráfico de boxplot utilizando seaborn
sns.boxplot(x=df[col], ax=ax)
# Establece el título de cada gráfico
ax.set_title('Boxplot South America by {}'.format(col))
# Muestra cada gráfico
plt.show()
df_num_south_america.head(1)
year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | landarea(km2) | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | 2000 | 95.78329 | 95.0 | 235.62 | 0.0 | 9.85 | 50.37 | 5.99 | 28.89 | 40.914955 | 18849.81 | 3.69 | 132270.0 | 12.283858 | -0.788999 | 7708.099115 | 2780400.0 |
df_num_south_america.shape
(211, 17)
df_num_south_america.describe()
year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | landarea(km2) | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
count | 211.000000 | 211.000000 | 211.000000 | 211.000000 | 2.110000e+02 | 211.000000 | 211.000000 | 211.000000 | 211.000000 | 211.000000 | 211.000000 | 211.000000 | 211.000000 | 211.000000 | 211.000000 | 211.000000 | 2.110000e+02 |
mean | 2009.952607 | 94.909058 | 84.102607 | 407.628246 | 1.177523e+08 | 32.320142 | 21.702844 | 3.296919 | 60.482891 | 60.880731 | 15559.032864 | 3.397630 | 78358.056732 | 29.222492 | 2.876778 | 6781.960312 | 1.566064e+06 |
std | 6.094309 | 6.642774 | 17.304756 | 361.159236 | 3.224945e+08 | 16.456433 | 29.913366 | 5.395672 | 118.431827 | 27.034127 | 5742.932373 | 0.720454 | 117395.860239 | 8.874942 | 4.917957 | 4108.842005 | 2.438872e+06 |
min | 2000.000000 | 72.113110 | 35.600000 | 29.400000 | 0.000000e+00 | 7.650000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 5345.900400 | 2.140000 | 1120.000005 | 10.167007 | -15.907951 | 954.400318 | 1.638200e+05 |
25% | 2005.000000 | 94.480274 | 75.250000 | 172.795000 | 4.600000e+05 | 18.305000 | 0.870000 | 0.000000 | 7.230000 | 41.926115 | 9918.690500 | 2.790000 | 4550.000191 | 22.571669 | 1.115547 | 3745.501842 | 2.149690e+05 |
50% | 2010.000000 | 97.194930 | 91.200000 | 300.290000 | 5.030000e+06 | 30.420000 | 8.360000 | 0.000000 | 22.500000 | 60.827293 | 16220.931000 | 3.360000 | 38290.000920 | 30.744228 | 3.283446 | 5919.012338 | 4.067520e+05 |
75% | 2015.000000 | 99.097675 | 99.000000 | 467.900000 | 9.778500e+07 | 43.025000 | 32.355000 | 6.525000 | 44.565000 | 83.380697 | 21089.465000 | 3.855000 | 79715.000155 | 36.035774 | 5.230067 | 8762.026548 | 1.285216e+06 |
max | 2020.000000 | 100.000000 | 100.000000 | 1410.110000 | 2.811680e+09 | 70.410000 | 136.580000 | 15.170000 | 520.010000 | 100.000000 | 25234.633000 | 6.250000 | 511619.995100 | 49.471880 | 43.479556 | 18703.860290 | 8.515770e+06 |
columns_with_outliers_south_america = []
for column in df_num_south_america.columns: if df_num_south_america[column].dtype == 'float64' or df_num_south_america[column].dtype == 'int64': Q1 = df_num_south_america[column].quantile(0.25) Q3 = df_num_south_america[column].quantile(0.75) IQR = Q3 - Q1
lower_limit = Q1 - 1.5 * IQR
upper_limit = Q3 + 1.5 * IQR
outliers = df_num_south_america[
(df_num_south_america[column] > upper_limit) |
(df_num_south_america[column] < lower_limit)
]
if not outliers.empty:
print(f"Columna: {column}")
print(outliers)
columns_with_outliers_south_america.append(column)
print("Columnas con outliers y número de outliers:") for column, num_outliers in columns_with_outliers_south_america.items(): print(f"{column}: {num_outliers}")
#Total number of outliers by column
column_outlier_total_south_america = {}
# Calcular el IQR para cada columna numérica y encontrar outliers
for column in df_num_south_america.columns:
if df_num_south_america[column].dtype == 'float64' or df_num_south_america[column].dtype == 'int64':
Q1 = df_num_south_america[column].quantile(0.25)
Q3 = df_num_south_america[column].quantile(0.75)
IQR = Q3 - Q1
lower_limit = Q1 - 1.5 * IQR
upper_limit = Q3 + 1.5 * IQR
outliers = df_num_south_america[
(df_num_south_america[column] > upper_limit) |
(df_num_south_america[column] < lower_limit)
]
num_outliers = len(outliers)
column_outlier_total_south_america[column] = num_outliers
# Mostrar el número total de outliers por columna
for column, num_outliers in column_outlier_total_south_america.items():
print(f"Número de outliers en {column}: {num_outliers}")
Número de outliers en year: 0 Número de outliers en access_to_electricity(%_of_population): 25 Número de outliers en access_to_clean_fuels_for_cooking: 2 Número de outliers en renewable_electricity_generating_capacity_per_capita: 26 Número de outliers en financial_flows_to_developing_countries(us$): 25 Número de outliers en renewable_energy_share_in_the_total_final_energy_consumption(%): 0 Número de outliers en electricity_from_fossil_fuels(twh): 18 Número de outliers en electricity_from_nuclear(twh): 0 Número de outliers en electricity_from_renewables(twh): 21 Número de outliers en low_carbonelectricity(%electricity): 0 Número de outliers en primary_energy_consumption_per_capita(kwh/person): 0 Número de outliers en energy_intensity_level_of_primary_energy(mj/$2017pppgdp): 5 Número de outliers en CO2_emissions_value_by_country_kt: 21 Número de outliers en renewables(%equivalent_primary_energy): 0 Número de outliers en gdp_growth: 12 Número de outliers en gdp_per_capita: 6 Número de outliers en landarea(km2): 21
#CHECKING DISTRIBUTION
for c in df_num_south_america.columns:
plt.figure(figsize=(10,4))
sns.histplot(df_num_south_america[c], kde= True)
plt.title(c)
plt.show()
#Comparison of general statistics of developed and undeveloped countries
df_num_europe.describe()
df_num_north_america.describe()
df_num_south_america.describe()
df_num_asia.describe()
El Objetivo de Desarrollo Sostenible 7 (Objetivo 7 o ODS 7) trata sobre alianzas para los objetivos y es uno de los 17 Objetivos de Desarrollo Sostenible (ODS) establecidos por las Naciones Unidas en 2015.
El principal problema que busca erradicar es la contaminación a causa de las energías fósiles y la ineficacia de los gobiernos para implementar energías limpias y renovables
El acceso a fuentes de energía modernas y sostenibles es esencial no solo para hacer frente al cambio climático, sino también para el crecimiento económico de los países.
La combinación de energía primaria de la región está dominada por los combustibles fósiles, y el carbón, el petróleo y el gas natural representan más del 80% del consumo total. Esta dependencia de los combustibles fósiles ha provocado altos niveles de emisiones de CO2, que contribuyen al cambio climático.
En los últimos años, ha habido una creciente conciencia sobre la necesidad de reducir las emisiones de CO2 en el Sudeste Asiático. Los gobiernos de toda la región han fijado objetivos ambiciosos para el despliegue de energía renovable y hay una inversión creciente en medidas de eficiencia energética.
Este análisis examinará las tendencias en el consumo de electricidad, el consumo de energía primaria y las emisiones de CO2 DE 4 continentes.
# concatenamos todos los dataframes en uno, recordando que cada dataframde esta dividdo por continente y ademas tien su data numerica en su respectivo dataset
dataframes = [continent_data['Europe'], continent_data['Asia'], continent_data['North America'], continent_data['South America']]
df_all = pd.concat(dataframes, ignore_index=True)
#Guardamos dataset limpio sin nulos
archivo_csv = 'df_all.csv'
df_all.to_csv(archivo_csv, index=False)
df_all.head()
entity | year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | density(P/km2) | landarea(km2) | continent | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | Albania | 2000 | 100.0 | 38.2 | 0.0 | 0.0 | 0.0 | 0.14 | 0.0 | 4.55 | 97.014930 | 9029.4375 | 4.13 | 3170.000000 | 0.0 | 6.946217 | 1126.683340 | 105 | 28748.0 | Europe |
1 | Albania | 2001 | 100.0 | 40.5 | 0.0 | 0.0 | 0.0 | 0.13 | 0.0 | 3.52 | 96.438354 | 8635.5320 | 3.89 | 3230.000000 | 0.0 | 8.293313 | 1281.659826 | 105 | 28748.0 | Europe |
2 | Albania | 2002 | 100.0 | 43.2 | 0.0 | 0.0 | 0.0 | 0.16 | 0.0 | 3.48 | 95.604390 | 9443.5550 | 4.10 | 3759.999990 | 0.0 | 4.536524 | 1425.124219 | 105 | 28748.0 | Europe |
3 | Albania | 2003 | 100.0 | 46.4 | 0.0 | 0.0 | 0.0 | 0.10 | 0.0 | 5.12 | 98.084300 | 10756.6120 | 3.80 | 4070.000172 | 0.0 | 5.528637 | 1846.120121 | 105 | 28748.0 | Europe |
4 | Albania | 2004 | 100.0 | 49.0 | 0.0 | 0.0 | 0.0 | 0.13 | 0.0 | 5.41 | 97.653430 | 11586.9510 | 3.96 | 4250.000000 | 0.0 | 5.514668 | 2373.581292 | 105 | 28748.0 | Europe |
df_all.shape
(2360, 20)
df_all.isnull().sum().sort_values(ascending=False)
entity 0 year 0 landarea(km2) 0 density(P/km2) 0 gdp_per_capita 0 gdp_growth 0 renewables(%equivalent_primary_energy) 0 CO2_emissions_value_by_country_kt 0 energy_intensity_level_of_primary_energy(mj/$2017pppgdp) 0 primary_energy_consumption_per_capita(kwh/person) 0 low_carbonelectricity(%electricity) 0 electricity_from_renewables(twh) 0 electricity_from_nuclear(twh) 0 electricity_from_fossil_fuels(twh) 0 renewable_energy_share_in_the_total_final_energy_consumption(%) 0 financial_flows_to_developing_countries(us$) 0 renewable_electricity_generating_capacity_per_capita 0 access_to_clean_fuels_for_cooking 0 access_to_electricity(%_of_population) 0 continent 0 dtype: int64
import matplotlib.pyplot as plt
import seaborn as sns
# Establecer el estilo de gráfico
sns.set(style='whitegrid')
# Crear un gráfico de líneas para cada continente
plt.figure(figsize=(10, 6))
# Iterar a través de los nombres únicos de los continentes en df_all
for continente in df_all['continent'].unique():
df_continente = df_all[df_all['continent'] == continente]
# Crear el gráfico de líneas para el continente actual
sns.lineplot(data=df_continente, x='year', y='primary_energy_consumption_per_capita(kwh/person)', label=continente)
# Personalizar el diseño del gráfico
plt.xlabel('Año', fontweight='bold')
plt.ylabel('Consumo de energía per cápita (kWh/person)', fontweight='bold')
plt.title('Consumo de energía per cápita por continente (2000 - 2020)', fontweight='bold')
plt.legend(title='Continente')
plt.xticks(rotation=45, ticks=df_all['year'].unique())
# Mostrar el gráfico
plt.show()
# Calcular el promedio de la columna por año y continente
df_avg = df_all.groupby(['year', 'continent'])['primary_energy_consumption_per_capita(kwh/person)'].mean().reset_index()
# Establecer el estilo de gráfico
sns.set(style='whitegrid')
# Crear un gráfico de líneas para cada continente
plt.figure(figsize=(10, 6))
# Iterar a través de los nombres únicos de los continentes en df_avg
for continente in df_avg['continent'].unique():
df_continente = df_avg[df_avg['continent'] == continente]
# Crear el gráfico de líneas para el continente actual
sns.lineplot(data=df_continente, x='year', y='primary_energy_consumption_per_capita(kwh/person)', label=continente)
# Personalizar el diseño del gráfico
plt.xlabel('Año', fontweight='bold')
plt.ylabel('Consumo promedio de energía per cápita (kWh/person)', fontweight='bold')
plt.title('Consumo promedio de energía per cápita por continente (2000 - 2020)', fontweight='bold')
plt.legend(title='Continente')
plt.xticks(rotation=45, ticks=df_avg['year'].unique())
# Mostrar el gráfico
plt.show()
#import matplotlib.pyplot as plt
import seaborn as sns
# Establecer el estilo de gráfico
sns.set(style='whitegrid')
# Crear un gráfico de líneas para cada continente
plt.figure(figsize=(10, 6))
# Iterar a través de los nombres únicos de los continentes en df_all
for continente in df_all['continent'].unique():
df_continente = df_all[df_all['continent'] == continente]
# Crear el gráfico de líneas para el continente actual
sns.lineplot(data=df_continente, x='year', y='electricity_from_fossil_fuels(twh)', label=continente)
# Personalizar el diseño del gráfico
plt.xlabel('Año', fontweight='bold')
plt.ylabel('electricity_from_fossil_fuels(twh)', fontweight='bold')
plt.title('electricity_from_fossil_fuels(twh) (2000 - 2020)', fontweight='bold')
plt.legend(title='Continente')
plt.xticks(rotation=45, ticks=df_all['year'].unique())
# Mostrar el gráfico
plt.show()
# Establecer el estilo de gráfico
sns.set(style='whitegrid')
# Crear un gráfico de líneas para cada continente
plt.figure(figsize=(10, 6))
# Iterar a través de los nombres únicos de los continentes en df_all
for continente in df_all['continent'].unique():
df_continente = df_all[df_all['continent'] == continente]
# Crear el gráfico de líneas para el continente actual
sns.lineplot(data=df_continente, x='year', y='electricity_from_nuclear(twh)', label=continente)
# Personalizar el diseño del gráfico
plt.xlabel('Año', fontweight='bold')
plt.ylabel('electricity_from_nuclear(twh)', fontweight='bold')
plt.title('electricity_from_nuclear(twh) (2000 - 2020)', fontweight='bold')
plt.legend(title='Continente')
plt.xticks(rotation=45, ticks=df_all['year'].unique())
# Mostrar el gráfico
plt.show()
## Electricity generated from renewables
sns.set(style='whitegrid')
# Crear un gráfico de líneas para cada continente
plt.figure(figsize=(10, 6))
# Iterar a través de los nombres únicos de los continentes en df_all
for continente in df_all['continent'].unique():
df_continente = df_all[df_all['continent'] == continente]
# Crear el gráfico de líneas para el continente actual
sns.lineplot(data=df_continente, x='year', y='electricity_from_renewables(twh)', label=continente)
# Personalizar el diseño del gráfico
plt.xlabel('Año', fontweight='bold')
plt.ylabel('electricity_from_renewables(twh)', fontweight='bold')
plt.title('electricity_from_renewables(twh) (2000 - 2020)', fontweight='bold')
plt.legend(title='Continente')
plt.xticks(rotation=45, ticks=df_all['year'].unique())
plt.show()
import matplotlib.pyplot as plt
# Filtrar los datos para incluir solo países de Asia en el año 2020
asia_2020 = df_all[(df_all['continent'] == 'Asia') & (df_all['year'] == 2020)]
# Ordenar los países por consumo de energía per cápita en orden descendente
top_10_asian_countries = asia_2020.sort_values(by='primary_energy_consumption_per_capita(kwh/person)', ascending=False).head(10)
# Crear un gráfico de barras
plt.figure(figsize=(7, 5))
plt.bar(top_10_asian_countries['entity'], top_10_asian_countries['primary_energy_consumption_per_capita(kwh/person)'], color='skyblue')
plt.ylabel('Consumo de energía per cápita (kWh/person)', fontweight='bold')
plt.xlabel('Países', fontweight='bold')
plt.title('Top 10 Países Asiáticos por Consumo de Energía per Cápita (2020)', fontweight='bold')
plt.xticks(rotation=45) # Rotar etiquetas del eje x para una mejor visualización
# Mostrar el gráfico
plt.tight_layout()
plt.show()
# Top 10 South America countries by energy consumption per capita as of 2020
sa_2020 = df_all[(df_all['continent'] == 'South America') & (df_all['year'] == 2020)]
top_10_sa = sa_2020.sort_values(by='primary_energy_consumption_per_capita(kwh/person)', ascending=False).head(10)
plt.figure(figsize=(7, 5))
bars = plt.bar(top_10_sa['entity'], top_10_sa['primary_energy_consumption_per_capita(kwh/person)'], color='skyblue')
plt.ylabel('Consumo de energía per cápita (kWh/person)', fontweight='bold')
plt.xlabel('Países', fontweight='bold')
plt.title('Top 10 Países Sudamericanos por Consumo de Energía per Cápita (2020)', fontweight='bold')
plt.xticks(rotation=45)
# Agregar etiquetas de valores a las barras
for bar in bars:
yval = bar.get_height()
plt.text(bar.get_x() + bar.get_width()/2, yval, f'{yval:.2f}', ha='center', va='bottom', fontsize=10, fontweight='bold')
plt.tight_layout()
plt.show()
# Filtrar los datos para incluir solo países de Asia en el año 2020
asia_2020 = df_all[(df_all['continent'] == 'Asia') & (df_all['year'] == 2020)]
# Ordenar los países por consumo de energía per cápita en orden descendente
top_10_asian_countries = asia_2020.sort_values(by='primary_energy_consumption_per_capita(kwh/person)', ascending=False).head(10)
# Crear un gráfico de barras
plt.figure(figsize=(10, 6))
bars = plt.bar(top_10_asian_countries['entity'], top_10_asian_countries['primary_energy_consumption_per_capita(kwh/person)'], color='skyblue')
plt.ylabel('Consumo de energía per cápita (kWh/person)', fontweight='bold')
plt.xlabel('Países', fontweight='bold')
plt.title('Top 10 Países Asiáticos por Consumo de Energía per Cápita (2020)', fontweight='bold')
plt.xticks(rotation=45) # Rotar etiquetas del eje x para una mejor visualización
# Agregar etiquetas de valores encima de las barras
for bar in bars:
yval = bar.get_height()
plt.text(bar.get_x() + bar.get_width()/2, yval, f'{yval:.2f}', ha='center', va='bottom', fontsize=10, fontweight='bold')
# Mostrar el gráfico
plt.tight_layout()
plt.show()
# Top 10 Europe countries by energy consumption per capita as of 2020
eu_2020 = df_all[(df_all['continent'] == 'Europe') & (df_all['year'] == 2020)]
top_10_eu = eu_2020.sort_values(by='primary_energy_consumption_per_capita(kwh/person)', ascending=False).head(10)
plt.figure(figsize=(7, 5))
bars = plt.bar(top_10_eu['entity'], top_10_eu['primary_energy_consumption_per_capita(kwh/person)'], color='skyblue')
plt.ylabel('Consumo de energía per cápita (kWh/person)', fontweight='bold')
plt.xlabel('Países', fontweight='bold')
plt.title('Top 10 Países Europeos por Consumo de Energía per Cápita (2020)', fontweight='bold')
plt.xticks(rotation=45)
# Agregar etiquetas de valores a las barras
for bar in bars:
yval = bar.get_height()
plt.text(bar.get_x() + bar.get_width()/2, yval, f'{yval:.2f}', ha='center', va='bottom', fontsize=10, fontweight='bold')
plt.tight_layout()
plt.show()
# Top 10 North America countries by energy consumption per capita as of 2020
na_2020 = df_all[(df_all['continent'] == 'North America') & (df_all['year'] == 2020)]
top_10_na = na_2020.sort_values(by='primary_energy_consumption_per_capita(kwh/person)', ascending=False).head(10)
plt.figure(figsize=(7, 5))
bars = plt.bar(top_10_na['entity'], top_10_na['primary_energy_consumption_per_capita(kwh/person)'], color='skyblue')
plt.ylabel('Consumo de energía per cápita (kWh/person)', fontweight='bold')
plt.xlabel('Países', fontweight='bold')
plt.title('Top 10 Países Norteamericanos por Consumo de Energía per Cápita (2020)', fontweight='bold')
plt.xticks(rotation=45)
# Agregar etiquetas de valores a las barras
for bar in bars:
yval = bar.get_height()
plt.text(bar.get_x() + bar.get_width()/2, yval, f'{yval:.2f}', ha='center', va='bottom', fontsize=10, fontweight='bold')
plt.tight_layout()
plt.show()
# 1. Agrupar los datos por año y continente y sumar las cantidades de electricidad
grouped_data = df_all.groupby(['year', 'continent'])[['electricity_from_renewables(twh)', 'electricity_from_fossil_fuels(twh)', 'electricity_from_nuclear(twh)']].sum().reset_index()
# 2. Crear un DataFrame con los datos agrupados para Asia
data_asia = grouped_data[grouped_data['continent'] == 'Asia']
# 3. Crear un gráfico de barras apiladas
fig, ax = plt.subplots()
data_asia.plot(x="year", y=['electricity_from_renewables(twh)', 'electricity_from_fossil_fuels(twh)', 'electricity_from_nuclear(twh)'],
kind="bar", stacked=True, ax=ax)
plt.xlabel('Años')
plt.ylabel('Electricidad generada en terawatt-hours')
plt.title('Electricidad por fuente para Asia')
plt.legend(['Renovables', 'Fósiles', 'Nuclear'], loc='upper left')
plt.grid(True)
# Muestra el gráfico
plt.show()
import matplotlib.pyplot as plt
# 1. Agrupar los datos por año y continente y sumar las cantidades de electricidad
grouped_data = df_all.groupby(['year', 'continent'])[['electricity_from_renewables(twh)', 'electricity_from_fossil_fuels(twh)', 'electricity_from_nuclear(twh)']].sum().reset_index()
# 2. Crear un DataFrame con los datos agrupados para Asia
data_asia = grouped_data[grouped_data['continent'] == 'Asia']
# 3. Filtrar los datos para los años 2010 y 2020
data_asia_filtered = data_asia[data_asia['year'].isin([2010, 2020])]
# 4. Crear un gráfico de barras apiladas
fig, ax = plt.subplots()
data_asia_filtered.plot(x="year", y=['electricity_from_renewables(twh)', 'electricity_from_fossil_fuels(twh)', 'electricity_from_nuclear(twh)'],
kind="bar", stacked=True, ax=ax)
plt.xlabel('Años')
plt.ylabel('Electricidad generada en terawatt-hours')
plt.title('Electricidad por fuente para Asia (2010 y 2020)')
plt.legend(['Renovables', 'Fósiles', 'Nuclear'], loc='upper left')
plt.grid(True)
# Agregar etiquetas de valores en las barras
for container in ax.containers:
ax.bar_label(container, fmt='%.2f', label_type='center')
# Muestra el gráfico
plt.show()
grouped_data = df_all.groupby(['year', 'continent'])[['electricity_from_renewables(twh)', 'electricity_from_fossil_fuels(twh)', 'electricity_from_nuclear(twh)']].sum().reset_index()
# 2. Crear un DataFrame con los datos agrupados para Asia
data_asia = grouped_data[grouped_data['continent'] == 'Asia']
# 3. Filtrar los datos para los años 2010 y 2020
data_asia_filtered = data_asia[data_asia['year'].isin([2000, 2010, 2020])]
# 4. Calcular los porcentajes
data_asia_filtered['total_electricity'] = data_asia_filtered[['electricity_from_renewables(twh)', 'electricity_from_fossil_fuels(twh)', 'electricity_from_nuclear(twh)']].sum(axis=1)
data_asia_filtered['renewable_percentage'] = (data_asia_filtered['electricity_from_renewables(twh)'] / data_asia_filtered['total_electricity']) * 100
data_asia_filtered['fossil_percentage'] = (data_asia_filtered['electricity_from_fossil_fuels(twh)'] / data_asia_filtered['total_electricity']) * 100
data_asia_filtered['nuclear_percentage'] = (data_asia_filtered['electricity_from_nuclear(twh)'] / data_asia_filtered['total_electricity']) * 100
# 5. Crear un gráfico de barras apiladas
fig, ax = plt.subplots()
data_asia_filtered.plot(x="year", y=['renewable_percentage', 'fossil_percentage', 'nuclear_percentage'],
kind="bar", stacked=True, ax=ax)
plt.xlabel('Años')
plt.ylabel('Porcentaje de electricidad generada')
plt.title('Porcentaje de Electricidad por fuente para Asia a (2000, 2010 y 2020)')
plt.legend(['Renovables', 'Fósiles', 'Nuclear'], loc='upper left')
plt.grid(True)
# Agregar etiquetas de valores en las barras
for container in ax.containers:
ax.bar_label(container, fmt='%.2f%%', label_type='center')
# Muestra el gráfico
plt.show()
C:\Users\danie\AppData\Local\Temp\ipykernel_17188\628181640.py:10: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy data_asia_filtered['total_electricity'] = data_asia_filtered[['electricity_from_renewables(twh)', 'electricity_from_fossil_fuels(twh)', 'electricity_from_nuclear(twh)']].sum(axis=1) C:\Users\danie\AppData\Local\Temp\ipykernel_17188\628181640.py:11: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy data_asia_filtered['renewable_percentage'] = (data_asia_filtered['electricity_from_renewables(twh)'] / data_asia_filtered['total_electricity']) * 100 C:\Users\danie\AppData\Local\Temp\ipykernel_17188\628181640.py:12: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy data_asia_filtered['fossil_percentage'] = (data_asia_filtered['electricity_from_fossil_fuels(twh)'] / data_asia_filtered['total_electricity']) * 100 C:\Users\danie\AppData\Local\Temp\ipykernel_17188\628181640.py:13: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy data_asia_filtered['nuclear_percentage'] = (data_asia_filtered['electricity_from_nuclear(twh)'] / data_asia_filtered['total_electricity']) * 100
# 1. Agrupar los datos por año y continente y sumar las cantidades de electricidad
grouped_data = df_all.groupby(['year', 'continent'])[['electricity_from_renewables(twh)', 'electricity_from_fossil_fuels(twh)', 'electricity_from_nuclear(twh)']].sum().reset_index()
# 2. Crear un DataFrame con los datos agrupados para Asia
data_sa = grouped_data[grouped_data['continent'] == 'South America']
# 3. Crear un gráfico de barras apiladas
fig, ax = plt.subplots()
data_sa.plot(x="year", y=['electricity_from_renewables(twh)', 'electricity_from_fossil_fuels(twh)', 'electricity_from_nuclear(twh)'],
kind="bar", stacked=True, ax=ax)
plt.xlabel('Años')
plt.ylabel('Electricidad geenrada (TWh)')
plt.title('Electricidad generada por distintas fuentes South America')
plt.legend(['Renovables', 'Fósiles', 'Nuclear'], loc='upper left')
plt.grid(True)
# Muestra el gráfico
plt.show()
import matplotlib.pyplot as plt
# 1. Agrupar los datos por año y continente y sumar las cantidades de electricidad
grouped_data = df_all.groupby(['year', 'continent'])[['electricity_from_renewables(twh)', 'electricity_from_fossil_fuels(twh)', 'electricity_from_nuclear(twh)']].sum().reset_index()
# 2. Crear un DataFrame con los datos agrupados para Asia
data_sa = grouped_data[grouped_data['continent'] == 'South America']
# 3. Filtrar los datos para los años 2010 y 2020
data_sa_f = data_sa[data_sa['year'].isin([2000, 2010, 2020])]
# 4. Crear un gráfico de barras apiladas
fig, ax = plt.subplots()
data_sa_f.plot(x="year", y=['electricity_from_renewables(twh)', 'electricity_from_fossil_fuels(twh)', 'electricity_from_nuclear(twh)'],
kind="bar", stacked=True, ax=ax)
plt.xlabel('Años')
plt.ylabel('Electricidad generada en terawatt-hours')
plt.title('Electricidad por fuente para South America (2000, 2010 y 2020)')
plt.legend(['Renovables', 'Fósiles', 'Nuclear'], loc='upper left')
plt.grid(True)
# Agregar etiquetas de valores en las barras
for container in ax.containers:
ax.bar_label(container, fmt='%.2f', label_type='center')
# Muestra el gráfico
plt.show()
# Filtrar los datos solo para los años 2000, 2010 y 2020
data_sa_filtered = data_sa[data_sa['year'].isin([2000, 2010, 2020])]
# 2. Crear un DataFrame con los datos agrupados para Asia
data_sa = grouped_data[grouped_data['continent'] == 'South America']
# Calcular los porcentajes
data_sa_filtered['total_electricity'] = data_sa_filtered[['electricity_from_renewables(twh)', 'electricity_from_fossil_fuels(twh)', 'electricity_from_nuclear(twh)']].sum(axis=1)
data_sa_filtered['renewable_percentage'] = (data_sa_filtered['electricity_from_renewables(twh)'] / data_sa_filtered['total_electricity']) * 100
data_sa_filtered['fossil_percentage'] = (data_sa_filtered['electricity_from_fossil_fuels(twh)'] / data_sa_filtered['total_electricity']) * 100
data_sa_filtered['nuclear_percentage'] = (data_sa_filtered['electricity_from_nuclear(twh)'] / data_sa_filtered['total_electricity']) * 100
# Crear un gráfico de barras apiladas
fig, ax = plt.subplots()
data_sa_filtered.plot(x="year", y=['renewable_percentage', 'fossil_percentage', 'nuclear_percentage'],
kind="bar", stacked=True, ax=ax)
plt.xlabel('Años')
plt.ylabel('Porcentaje de electricidad generada')
plt.title('Porcentaje de Electricidad por fuente para South America en (2000, 2010 y 2020)')
plt.legend(['Renovables', 'Fósiles', 'Nuclear'], loc='upper left')
plt.grid(True)
# Agregar etiquetas de valores en las barras
for container in ax.containers:
ax.bar_label(container, fmt='%.2f%%', label_type='center')
# Muestra el gráfico
plt.show()
# 1. Agrupar los datos por año y continente y sumar las cantidades de electricidad
grouped_data = df_all.groupby(['year', 'continent'])[['electricity_from_renewables(twh)', 'electricity_from_fossil_fuels(twh)', 'electricity_from_nuclear(twh)']].sum().reset_index()
# 2. Crear un DataFrame con los datos agrupados para Asia
data_eu = grouped_data[grouped_data['continent'] == 'Europe']
# 3. Crear un gráfico de barras apiladas
fig, ax = plt.subplots()
data_eu.plot(x="year", y=['electricity_from_renewables(twh)', 'electricity_from_fossil_fuels(twh)', 'electricity_from_nuclear(twh)'],
kind="bar", stacked=True, ax=ax)
plt.xlabel('Años')
plt.ylabel('Electricidad geenrada (TWh)')
plt.title('Electricidad generada por distintas fuentes Europe')
plt.legend(['Renovables', 'Fósiles', 'Nuclear'], loc='upper left')
plt.grid(True)
# Muestra el gráfico
plt.show()
import matplotlib.pyplot as plt
# 1. Agrupar los datos por año y continente y sumar las cantidades de electricidad
grouped_data = df_all.groupby(['year', 'continent'])[['electricity_from_renewables(twh)', 'electricity_from_fossil_fuels(twh)', 'electricity_from_nuclear(twh)']].sum().reset_index()
# 2. Crear un DataFrame con los datos agrupados para Asia
data_asia = grouped_data[grouped_data['continent'] == 'Europe']
# 3. Filtrar los datos para los años 2010 y 2020
data_asia_filtered = data_asia[data_asia['year'].isin([2010, 2020])]
# 4. Crear un gráfico de barras apiladas
fig, ax = plt.subplots()
data_asia_filtered.plot(x="year", y=['electricity_from_renewables(twh)', 'electricity_from_fossil_fuels(twh)', 'electricity_from_nuclear(twh)'],
kind="bar", stacked=True, ax=ax)
plt.xlabel('Años')
plt.ylabel('Electricidad generada en terawatt-hours')
plt.title('Electricidad por fuente para Europe (2010 y 2020)')
plt.legend(['Renovables', 'Fósiles', 'Nuclear'], loc='upper left')
plt.grid(True)
# Agregar etiquetas de valores en las barras
for container in ax.containers:
ax.bar_label(container, fmt='%.2f', label_type='center')
# Muestra el gráfico
plt.show()
# 1. Agrupar los datos por año y continente y sumar las cantidades de electricidad
grouped_data = df_all.groupby(['year', 'continent'])[['electricity_from_renewables(twh)', 'electricity_from_fossil_fuels(twh)', 'electricity_from_nuclear(twh)']].sum().reset_index()
# 2. Crear un DataFrame con los datos agrupados para Asia
data_asia = grouped_data[grouped_data['continent'] == 'Europe']
# 3. Filtrar los datos para los años 2010 y 2020
data_asia_filtered = data_asia[data_asia['year'].isin([2000, 2010, 2020])]
# 4. Calcular los porcentajes
data_asia_filtered['total_electricity'] = data_asia_filtered[['electricity_from_renewables(twh)', 'electricity_from_fossil_fuels(twh)', 'electricity_from_nuclear(twh)']].sum(axis=1)
data_asia_filtered['renewable_percentage'] = (data_asia_filtered['electricity_from_renewables(twh)'] / data_asia_filtered['total_electricity']) * 100
data_asia_filtered['fossil_percentage'] = (data_asia_filtered['electricity_from_fossil_fuels(twh)'] / data_asia_filtered['total_electricity']) * 100
data_asia_filtered['nuclear_percentage'] = (data_asia_filtered['electricity_from_nuclear(twh)'] / data_asia_filtered['total_electricity']) * 100
# 5. Crear un gráfico de barras apiladas
fig, ax = plt.subplots()
data_asia_filtered.plot(x="year", y=['renewable_percentage', 'fossil_percentage', 'nuclear_percentage'],
kind="bar", stacked=True, ax=ax)
plt.xlabel('Años')
plt.ylabel('Porcentaje de electricidad generada')
plt.title('Porcentaje de Electricidad por fuente para Europe a (2010 y 2020)')
plt.legend(['Renovables', 'Fósiles', 'Nuclear'], loc='upper left')
plt.grid(True)
# Agregar etiquetas de valores en las barras
for container in ax.containers:
ax.bar_label(container, fmt='%.2f%%', label_type='center')
# Muestra el gráfico
plt.show()
C:\Users\danie\AppData\Local\Temp\ipykernel_17188\3669201025.py:11: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy data_asia_filtered['total_electricity'] = data_asia_filtered[['electricity_from_renewables(twh)', 'electricity_from_fossil_fuels(twh)', 'electricity_from_nuclear(twh)']].sum(axis=1) C:\Users\danie\AppData\Local\Temp\ipykernel_17188\3669201025.py:12: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy data_asia_filtered['renewable_percentage'] = (data_asia_filtered['electricity_from_renewables(twh)'] / data_asia_filtered['total_electricity']) * 100 C:\Users\danie\AppData\Local\Temp\ipykernel_17188\3669201025.py:13: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy data_asia_filtered['fossil_percentage'] = (data_asia_filtered['electricity_from_fossil_fuels(twh)'] / data_asia_filtered['total_electricity']) * 100 C:\Users\danie\AppData\Local\Temp\ipykernel_17188\3669201025.py:14: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy data_asia_filtered['nuclear_percentage'] = (data_asia_filtered['electricity_from_nuclear(twh)'] / data_asia_filtered['total_electricity']) * 100
# 1. Agrupar los datos por año y continente y sumar las cantidades de electricidad
grouped_data = df_all.groupby(['year', 'continent'])[['electricity_from_renewables(twh)', 'electricity_from_fossil_fuels(twh)', 'electricity_from_nuclear(twh)']].sum().reset_index()
# 2. Crear un DataFrame con los datos agrupados para Asia
data_na = grouped_data[grouped_data['continent'] == 'North America']
# 3. Crear un gráfico de barras apiladas
fig, ax = plt.subplots()
data_na.plot(x="year", y=['electricity_from_renewables(twh)', 'electricity_from_fossil_fuels(twh)', 'electricity_from_nuclear(twh)'],
kind="bar", stacked=True, ax=ax)
plt.xlabel('Años')
plt.ylabel('Electricidad geenrada (TWh)')
plt.title('Electricidad generada por distintas fuentes North America')
plt.legend(['Renovables', 'Fósiles', 'Nuclear'], loc='upper left')
plt.grid(True)
# Muestra el gráfico
plt.show()
import matplotlib.pyplot as plt
# 1. Agrupar los datos por año y continente y sumar las cantidades de electricidad
grouped_data = df_all.groupby(['year', 'continent'])[['electricity_from_renewables(twh)', 'electricity_from_fossil_fuels(twh)', 'electricity_from_nuclear(twh)']].sum().reset_index()
# 2. Crear un DataFrame con los datos agrupados para Asia
data_na = grouped_data[grouped_data['continent'] == 'North America']
# 3. Filtrar los datos para los años 2010 y 2020
data_na_filtered = data_na[data_na['year'].isin([2010, 2020])]
# 4. Crear un gráfico de barras apiladas
fig, ax = plt.subplots()
data_na_filtered.plot(x="year", y=['electricity_from_renewables(twh)', 'electricity_from_fossil_fuels(twh)', 'electricity_from_nuclear(twh)'],
kind="bar", stacked=True, ax=ax)
plt.xlabel('Años')
plt.ylabel('Electricidad generada en terawatt-hours')
plt.title('Electricidad por fuente para North America(2010 y 2020)')
plt.legend(['Renovables', 'Fósiles', 'Nuclear'], loc='upper left')
plt.grid(True)
# Agregar etiquetas de valores en las barras
for container in ax.containers:
ax.bar_label(container, fmt='%.2f', label_type='center')
# Muestra el gráfico
plt.show()
# 1. Agrupar los datos por año y continente y sumar las cantidades de electricidad
grouped_data = df_all.groupby(['year', 'continent'])[['electricity_from_renewables(twh)', 'electricity_from_fossil_fuels(twh)', 'electricity_from_nuclear(twh)']].sum().reset_index()
# 2. Crear un DataFrame con los datos agrupados para Asia
data_na = grouped_data[grouped_data['continent'] == 'North America']
# 3. Filtrar los datos para los años 2010 y 2020
data_na_filtered = data_na[data_na['year'].isin([2000, 2010, 2020])]
# 4. Calcular los porcentajes
data_na_filtered['total_electricity'] = data_na_filtered[['electricity_from_renewables(twh)', 'electricity_from_fossil_fuels(twh)', 'electricity_from_nuclear(twh)']].sum(axis=1)
data_na_filtered['renewable_percentage'] = (data_na_filtered['electricity_from_renewables(twh)'] / data_na_filtered['total_electricity']) * 100
data_na_filtered['fossil_percentage'] = (data_na_filtered['electricity_from_fossil_fuels(twh)'] / data_na_filtered['total_electricity']) * 100
data_na_filtered['nuclear_percentage'] = (data_na_filtered['electricity_from_nuclear(twh)'] / data_na_filtered['total_electricity']) * 100
# 5. Crear un gráfico de barras apiladas
fig, ax = plt.subplots()
data_na_filtered.plot(x="year", y=['renewable_percentage', 'fossil_percentage', 'nuclear_percentage'],
kind="bar", stacked=True, ax=ax)
plt.xlabel('Años')
plt.ylabel('Porcentaje de electricidad generada')
plt.title('Porcentaje de Electricidad por fuente para North America (2010 y 2020)')
plt.legend(['Renovables', 'Fósiles', 'Nuclear'], loc='upper left')
plt.grid(True)
# Agregar etiquetas de valores en las barras
for container in ax.containers:
ax.bar_label(container, fmt='%.2f%%', label_type='center')
# Muestra el gráfico
plt.show()
C:\Users\danie\AppData\Local\Temp\ipykernel_17188\2363528489.py:11: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy data_na_filtered['total_electricity'] = data_na_filtered[['electricity_from_renewables(twh)', 'electricity_from_fossil_fuels(twh)', 'electricity_from_nuclear(twh)']].sum(axis=1) C:\Users\danie\AppData\Local\Temp\ipykernel_17188\2363528489.py:12: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy data_na_filtered['renewable_percentage'] = (data_na_filtered['electricity_from_renewables(twh)'] / data_na_filtered['total_electricity']) * 100 C:\Users\danie\AppData\Local\Temp\ipykernel_17188\2363528489.py:13: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy data_na_filtered['fossil_percentage'] = (data_na_filtered['electricity_from_fossil_fuels(twh)'] / data_na_filtered['total_electricity']) * 100 C:\Users\danie\AppData\Local\Temp\ipykernel_17188\2363528489.py:14: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy data_na_filtered['nuclear_percentage'] = (data_na_filtered['electricity_from_nuclear(twh)'] / data_na_filtered['total_electricity']) * 100
## Electricity generated from renewables
sns.set(style='whitegrid')
# Crear un gráfico de líneas para cada continente
plt.figure(figsize=(10, 6))
# Iterar a través de los nombres únicos de los continentes en df_all
for continente in df_all['continent'].unique():
df_continente = df_all[df_all['continent'] == continente]
# Crear el gráfico de líneas para el continente actual
sns.lineplot(data=df_continente, x='year', y='gdp_growth', label=continente)
# Personalizar el diseño del gráfico
plt.xlabel('Año', fontweight='bold')
plt.ylabel('gdp_growth', fontweight='bold')
plt.title('gdp_growth (2000 - 2020)', fontweight='bold')
plt.legend(title='Continente')
plt.xticks(rotation=45, ticks=df_all['year'].unique())
plt.show()
df_all.head(1)
entity | year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | density(P/km2) | landarea(km2) | continent | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | Albania | 2000 | 100.0 | 38.2 | 0.0 | 0.0 | 0.0 | 0.14 | 0.0 | 4.55 | 97.01493 | 9029.4375 | 4.13 | 3170.0 | 0.0 | 6.946217 | 1126.68334 | 105 | 28748.0 | Europe |
## Electricity generated from renewables
sns.set(style='whitegrid')
# Crear un gráfico de líneas para cada continente
plt.figure(figsize=(10, 6))
# Iterar a través de los nombres únicos de los continentes en df_all
for continente in df_all['continent'].unique():
df_continente = df_all[df_all['continent'] == continente]
# Crear el gráfico de líneas para el continente actual
sns.lineplot(data=df_continente, x='year', y='gdp_per_capita', label=continente)
# Personalizar el diseño del gráfico
plt.xlabel('Año', fontweight='bold')
plt.ylabel('gdp_per_capita', fontweight='bold')
plt.title('gdp_per_capita (2000 - 2020)', fontweight='bold')
plt.legend(title='Continente')
plt.xticks(rotation=45, ticks=df_all['year'].unique())
plt.show()
Primary energy is energy that is extracted directly from its sources, before any transformation or conversion. The energy that is available in its raw or unprocessed form is used to power various sectors and industry
primary_energy_all = df_all[['entity','year','primary_energy_consumption_per_capita(kwh/person)','energy_intensity_level_of_primary_energy(mj/$2017pppgdp)','renewables(%equivalent_primary_energy)','gdp_per_capita']]
primary_energy_all.head()
entity | year | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | renewables(%equivalent_primary_energy) | gdp_per_capita | |
---|---|---|---|---|---|---|
0 | Albania | 2000 | 9029.4375 | 4.13 | 0.0 | 1126.683340 |
1 | Albania | 2001 | 8635.5320 | 3.89 | 0.0 | 1281.659826 |
2 | Albania | 2002 | 9443.5550 | 4.10 | 0.0 | 1425.124219 |
3 | Albania | 2003 | 10756.6120 | 3.80 | 0.0 | 1846.120121 |
4 | Albania | 2004 | 11586.9510 | 3.96 | 0.0 | 2373.581292 |
df_all.head(2)
entity | year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | density(P/km2) | landarea(km2) | continent | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | Albania | 2000 | 100.0 | 38.2 | 0.0 | 0.0 | 0.0 | 0.14 | 0.0 | 4.55 | 97.014930 | 9029.4375 | 4.13 | 3170.0 | 0.0 | 6.946217 | 1126.683340 | 105 | 28748.0 | Europe |
1 | Albania | 2001 | 100.0 | 40.5 | 0.0 | 0.0 | 0.0 | 0.13 | 0.0 | 3.52 | 96.438354 | 8635.5320 | 3.89 | 3230.0 | 0.0 | 8.293313 | 1281.659826 | 105 | 28748.0 | Europe |
import plotly.subplots as sp
import plotly.graph_objs as go
# Selecciona el país que deseas mostrar
country = 'Chile'
# Filtra el DataFrame para el país seleccionado
country_data = df_all[df_all['entity'] == country]
# Especifica las columnas que deseas comparar
column_1 = 'primary_energy_consumption_per_capita(kwh/person)'
column_2 = 'gdp_per_capita'
# Crea el gráfico de líneas con dos ejes
fig = sp.make_subplots(specs=[[{"secondary_y": True}]])
fig.add_trace(go.Scatter(x=country_data['year'], y=country_data[column_1], name=f'{country} - {column_1}', mode='lines+markers'), secondary_y=False)
fig.add_trace(go.Scatter(x=country_data['year'], y=country_data[column_2], name=f'{country} - {column_2}', mode='lines+markers'), secondary_y=True)
fig.update_layout(title=f'{column_1} & {column_2} - {country}', xaxis_title='Year', yaxis_title='Value')
fig.update_yaxes(title_text=column_1, secondary_y=False)
fig.update_yaxes(title_text=column_2, secondary_y=True)
fig.show()
# Selecciona el país que deseas mostrar
country = 'Qatar'
# Filtra el DataFrame para el país seleccionado
country_data = df_all[df_all['entity'] == country]
# Especifica las columnas que deseas comparar
column_1 = 'primary_energy_consumption_per_capita(kwh/person)'
column_2 = 'gdp_per_capita'
# Crea el gráfico de líneas con dos ejes
fig = sp.make_subplots(specs=[[{"secondary_y": True}]])
fig.add_trace(go.Scatter(x=country_data['year'], y=country_data[column_1], name=f'{country} - {column_1}', mode='lines+markers'), secondary_y=False)
fig.add_trace(go.Scatter(x=country_data['year'], y=country_data[column_2], name=f'{country} - {column_2}', mode='lines+markers'), secondary_y=True)
fig.update_layout(title=f'{column_1} & {column_2} - {country}', xaxis_title='Year', yaxis_title='Value')
fig.update_yaxes(title_text=column_1, secondary_y=False)
fig.update_yaxes(title_text=column_2, secondary_y=True)
fig.show()
# Selecciona el país que deseas mostrar
country = 'Iceland'
# Filtra el DataFrame para el país seleccionado
country_data = df_all[df_all['entity'] == country]
# Especifica las columnas que deseas comparar
column_1 = 'primary_energy_consumption_per_capita(kwh/person)'
column_2 = 'gdp_per_capita'
# Crea el gráfico de líneas con dos ejes
fig = sp.make_subplots(specs=[[{"secondary_y": True}]])
fig.add_trace(go.Scatter(x=country_data['year'], y=country_data[column_1], name=f'{country} - {column_1}', mode='lines+markers'), secondary_y=False)
fig.add_trace(go.Scatter(x=country_data['year'], y=country_data[column_2], name=f'{country} - {column_2}', mode='lines+markers'), secondary_y=True)
fig.update_layout(title=f'{column_1} & {column_2} - {country}', xaxis_title='Year', yaxis_title='Value')
fig.update_yaxes(title_text=column_1, secondary_y=False)
fig.update_yaxes(title_text=column_2, secondary_y=True)
fig.show()
# Selecciona el país que deseas mostrar
country = 'Trinidad and Tobago'
# Filtra el DataFrame para el país seleccionado
country_data = df_all[df_all['entity'] == country]
# Especifica las columnas que deseas comparar
column_1 = 'primary_energy_consumption_per_capita(kwh/person)'
column_2 = 'gdp_per_capita'
# Crea el gráfico de líneas con dos ejes
fig = sp.make_subplots(specs=[[{"secondary_y": True}]])
fig.add_trace(go.Scatter(x=country_data['year'], y=country_data[column_1], name=f'{country} - {column_1}', mode='lines+markers'), secondary_y=False)
fig.add_trace(go.Scatter(x=country_data['year'], y=country_data[column_2], name=f'{country} - {column_2}', mode='lines+markers'), secondary_y=True)
fig.update_layout(title=f'{column_1} & {column_2} - {country}', xaxis_title='Year', yaxis_title='Value')
fig.update_yaxes(title_text=column_1, secondary_y=False)
fig.update_yaxes(title_text=column_2, secondary_y=True)
fig.show()
# Selecciona los países que deseas mostrar
countries = ['Chile', 'Iceland', 'Trinidad and Tobago', 'Qatar']
# Filtra el DataFrame para los países seleccionados
selected_data = df_all[df_all['entity'].isin(countries)]
# Especifica la columna que deseas comparar
column = 'energy_intensity_level_of_primary_energy(mj/$2017pppgdp)'
# Crea el gráfico de líneas con dos ejes
fig = sp.make_subplots(specs=[[{"secondary_y": True}]])
for country in countries:
country_data = selected_data[selected_data['entity'] == country]
fig.add_trace(go.Scatter(x=country_data['year'], y=country_data[column], name=f'{country} - {column}', mode='lines+markers'), secondary_y=False)
fig.update_layout(title=f'{column} por país', xaxis_title='Year', yaxis_title='Valor')
fig.update_yaxes(title_text=column, secondary_y=False)
fig.show()
Energy intensity level of primary energy. Energy intensity is defined as the energy supplied to the economy per unit value of economic output (megajoules per constant 2011 purchasing power parity GDP).TARGET 7.3 By 2030, double the global rate of improvement in energy efficiency. La variable "Energy intensity level of primary energy, MJ/constant 2011 PPP GDP" (Nivel de intensidad energética de la energía primaria, MJ/PIB constante de 2011 en paridad de poder adquisitivo) es una medida que evalúa la eficiencia energética de un país en relación con su Producto Interno Bruto (PIB) ajustado a la paridad de poder adquisitivo (PPA) en términos constantes de 2011. Esta medida se utiliza para cuantificar la cantidad de energía primaria que un país consume para producir un cierto valor de PIB.
En términos simples, indica cuánta energía se necesita para generar una unidad de valor económico en el país. Cuanto menor sea el valor de esta variable, mayor será la eficiencia energética del país, ya que significa que está utilizando menos energía para producir su PIB.
El hecho de que se ajuste a términos constantes de 2011 y se exprese en paridad de poder adquisitivo (PPA) significa que se han tenido en cuenta las variaciones en los precios y el valor del dinero a lo largo del tiempo y entre los países para hacer comparaciones más precisas y equitativas en términos de eficiencia energética.
En resumen, una menor "Energy intensity level of primary energy, MJ/constant 2011 PPP GDP" indica una mayor eficiencia energética en la producción económica de un país, mientras que un valor más alto sugiere que el país consume más energía para generar su PIB.
## Electricity generated from renewables
sns.set(style='whitegrid')
# Crear un gráfico de líneas para cada continente
plt.figure(figsize=(10, 6))
# Iterar a través de los nombres únicos de los continentes en df_all
for continente in df_all['continent'].unique():
df_continente = df_all[df_all['continent'] == continente]
# Crear el gráfico de líneas para el continente actual
sns.lineplot(data=df_continente, x='year', y='energy_intensity_level_of_primary_energy(mj/$2017pppgdp)', label=continente)
# Personalizar el diseño del gráfico
plt.xlabel('Año', fontweight='bold')
plt.ylabel('', fontweight='bold')
plt.title('energy_intensity_level_of_primary_energy(mj/$2017pppgdp) (2000 - 2020)', fontweight='bold')
plt.legend(title='Continente')
plt.xticks(rotation=45, ticks=df_all['year'].unique())
plt.show()
# Supongamos que tienes un DataFrame df_all con columnas 'year', 'continent', y 'primary_energy_consumption(kwh)'.
import pandas as pd
# Supongamos que tienes un DataFrame df_all con las columnas mencionadas.
# 1. Calcular la intensidad energética primaria en megajulios por PIB
df_all['primary_energy_intensity'] = df_all['energy_intensity_level_of_primary_energy(mj/$2017pppgdp)']
# 2. Calcular el Improvement Rate para cada período
def calculate_improvement_rate(df, start_year, end_year):
start_intensity = df[df['year'] == start_year]['primary_energy_intensity'].values[0]
end_intensity = df[df['year'] == end_year]['primary_energy_intensity'].values[0]
improvement_rate = ((end_intensity - start_intensity) / start_intensity) * 100
return improvement_rate
# Calcular Improvement Rate para los períodos específicos
improvement_2000_2010 = calculate_improvement_rate(df_all, 2000, 2010)
improvement_2010_2020 = calculate_improvement_rate(df_all, 2010, 2020)
print(f"Improvement Rate 2000-2010: {improvement_2000_2010:.2f}%")
print(f"Improvement Rate 2010-2020: {improvement_2010_2020:.2f}%")
Improvement Rate 2000-2010: -30.99% Improvement Rate 2010-2020: -12.28%
df_all['primary_energy_intensity'] = df_all['energy_intensity_level_of_primary_energy(mj/$2017pppgdp)']
# 2. Calcular el Improvement Rate por continente y período
def calculate_improvement_rate(df, start_year, end_year):
df_filtered = df[(df['year'] >= start_year) & (df['year'] <= end_year)]
grouped = df_filtered.groupby(['continent', 'year'])['primary_energy_intensity'].mean().unstack()
improvement_rate = ((grouped[end_year] - grouped[start_year]) / grouped[start_year]) * 100
return improvement_rate
# Calcular Improvement Rate por continente para los períodos específicos
improvement_2000_2010 = calculate_improvement_rate(df_all, 2000, 2010)
improvement_2010_2020 = calculate_improvement_rate(df_all, 2010, 2020)
print("Improvement Rate 2000-2010:")
print(improvement_2000_2010)
print("Improvement Rate 2010-2016:")
print(improvement_2010_2020)
Improvement Rate 2000-2010: continent Asia -25.044134 Europe -16.601000 North America -2.079736 South America -12.939024 dtype: float64 Improvement Rate 2010-2016: continent Asia -16.028495 Europe -22.138858 North America -15.861185 South America 2.989214 dtype: float64
df_all.head(3)
entity | year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | density(P/km2) | landarea(km2) | continent | primary_energy_intensity | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | Albania | 2000 | 100.0 | 38.2 | 0.0 | 0.0 | 0.0 | 0.14 | 0.0 | 4.55 | 97.014930 | 9029.4375 | 4.13 | 3170.00000 | 0.0 | 6.946217 | 1126.683340 | 105 | 28748.0 | Europe | 4.13 |
1 | Albania | 2001 | 100.0 | 40.5 | 0.0 | 0.0 | 0.0 | 0.13 | 0.0 | 3.52 | 96.438354 | 8635.5320 | 3.89 | 3230.00000 | 0.0 | 8.293313 | 1281.659826 | 105 | 28748.0 | Europe | 3.89 |
2 | Albania | 2002 | 100.0 | 43.2 | 0.0 | 0.0 | 0.0 | 0.16 | 0.0 | 3.48 | 95.604390 | 9443.5550 | 4.10 | 3759.99999 | 0.0 | 4.536524 | 1425.124219 | 105 | 28748.0 | Europe | 4.10 |
## Electricity generated from renewables
sns.set(style='whitegrid')
# Crear un gráfico de líneas para cada continente
plt.figure(figsize=(10, 6))
# Iterar a través de los nombres únicos de los continentes en df_all
for continente in df_all['continent'].unique():
df_continente = df_all[df_all['continent'] == continente]
# Crear el gráfico de líneas para el continente actual
sns.lineplot(data=df_continente, x='year', y='CO2_emissions_value_by_country_kt', label=continente)
# Personalizar el diseño del gráfico
plt.xlabel('Año', fontweight='bold')
plt.ylabel('', fontweight='bold')
plt.title('CO2 Emissions by continent (2000 - 2020)', fontweight='bold')
plt.legend(title='Continente')
plt.xticks(rotation=45, ticks=df_all['year'].unique())
plt.show()
# Top 10 North America countries by C02
nac02_2020 = df_all[(df_all['continent'] == 'North America') & (df_all['year'] == 2020)]
top_10_na_c02 = nac02_2020.sort_values(by='CO2_emissions_value_by_country_kt', ascending=False).head(10)
plt.figure(figsize=(7, 5))
plt.bar(top_10_na_c02['entity'], top_10_na_c02['CO2_emissions_value_by_country_kt'], color='skyblue')
plt.ylabel('CO2 Emissiobs value by kt', fontweight='bold')
plt.xlabel('Países', fontweight='bold')
plt.title('Top 10 Países Norteamericanos por C02 Emissions (2020)', fontweight='bold')
plt.xticks(rotation=45)
plt.tight_layout()
plt.show()
euc02_2020 = df_all[(df_all['continent'] == 'Europe') & (df_all['year'] == 2019)]
top_10_eu_c02 = euc02_2020.sort_values(by='CO2_emissions_value_by_country_kt', ascending=False).head(10)
plt.figure(figsize=(7, 5))
plt.bar(top_10_eu_c02['entity'], top_10_eu_c02['CO2_emissions_value_by_country_kt'], color='skyblue')
plt.ylabel('CO2 Emissiobs value by kt', fontweight='bold')
plt.xlabel('Países', fontweight='bold')
plt.title('Top 10 Países Europeos por C02 Emissions (2020)', fontweight='bold')
plt.xticks(rotation=45)
plt.tight_layout()
plt.show()
sac02_2020 = df_all[(df_all['continent'] == 'South America') & (df_all['year'] == 2020)]
top_10_sa_c02 = sac02_2020.sort_values(by='CO2_emissions_value_by_country_kt', ascending=False).head(10)
plt.figure(figsize=(7, 5))
plt.bar(top_10_sa_c02['entity'], top_10_sa_c02['CO2_emissions_value_by_country_kt'], color='skyblue')
plt.ylabel('CO2 Emissiobs value by kt', fontweight='bold')
plt.xlabel('Países', fontweight='bold')
plt.title('Top 10 Países Sudamericanos por C02 Emissions (2020)', fontweight='bold')
plt.xticks(rotation=45)
plt.tight_layout()
plt.show()
asiac02_2020 = df_all[(df_all['continent'] == 'Asia') & (df_all['year'] == 2020)]
top_10_asia_c02 = asiac02_2020.sort_values(by='CO2_emissions_value_by_country_kt', ascending=False).head(10)
plt.figure(figsize=(7, 5))
plt.bar(top_10_asia_c02['entity'], top_10_asia_c02['CO2_emissions_value_by_country_kt'], color='skyblue')
plt.ylabel('CO2 Emissiobs value by kt', fontweight='bold')
plt.xlabel('Países', fontweight='bold')
plt.title('Top 10 Países Asiáticos Emissions (2020)', fontweight='bold')
plt.xticks(rotation=45)
plt.tight_layout()
plt.show()
# Selecciona el país que deseas mostrar
country = 'United States'
# Filtra el DataFrame para el país seleccionado
country_data = df_all[df_all['entity'] == country]
# Especifica las columnas que deseas comparar
column_1 = 'CO2_emissions_value_by_country_kt'
column_2 = 'gdp_growth'
# Crea el gráfico de líneas con dos ejes
fig = sp.make_subplots(specs=[[{"secondary_y": True}]])
fig.add_trace(go.Scatter(x=country_data['year'], y=country_data[column_1], name=f'{country} - {column_1}', mode='lines+markers'), secondary_y=False)
fig.add_trace(go.Scatter(x=country_data['year'], y=country_data[column_2], name=f'{country} - {column_2}', mode='lines+markers'), secondary_y=True)
fig.update_layout(title=f'{column_1} & {column_2} - {country}', xaxis_title='Year', yaxis_title='Value')
fig.update_yaxes(title_text=column_1, secondary_y=False)
fig.update_yaxes(title_text=column_2, secondary_y=True)
fig.show()
# Selecciona el país que deseas mostrar
country = 'Brazil'
# Filtra el DataFrame para el país seleccionado
country_data = df_all[df_all['entity'] == country]
# Especifica las columnas que deseas comparar
column_1 = 'CO2_emissions_value_by_country_kt'
column_2 = 'gdp_growth'
# Crea el gráfico de líneas con dos ejes
fig = sp.make_subplots(specs=[[{"secondary_y": True}]])
fig.add_trace(go.Scatter(x=country_data['year'], y=country_data[column_1], name=f'{country} - {column_1}', mode='lines+markers'), secondary_y=False)
fig.add_trace(go.Scatter(x=country_data['year'], y=country_data[column_2], name=f'{country} - {column_2}', mode='lines+markers'), secondary_y=True)
fig.update_layout(title=f'{column_1} & {column_2} - {country}', xaxis_title='Year', yaxis_title='Value')
fig.update_yaxes(title_text=column_1, secondary_y=False)
fig.update_yaxes(title_text=column_2, secondary_y=True)
fig.show()
# Selecciona el país que deseas mostrar
country = 'China'
# Filtra el DataFrame para el país seleccionado
country_data = df_all[df_all['entity'] == country]
# Especifica las columnas que deseas comparar
column_1 = 'CO2_emissions_value_by_country_kt'
column_2 = 'gdp_growth'
# Crea el gráfico de líneas con dos ejes
fig = sp.make_subplots(specs=[[{"secondary_y": True}]])
fig.add_trace(go.Scatter(x=country_data['year'], y=country_data[column_1], name=f'{country} - {column_1}', mode='lines+markers'), secondary_y=False)
fig.add_trace(go.Scatter(x=country_data['year'], y=country_data[column_2], name=f'{country} - {column_2}', mode='lines+markers'), secondary_y=True)
fig.update_layout(title=f'{column_1} & {column_2} - {country}', xaxis_title='Year', yaxis_title='Value')
fig.update_yaxes(title_text=column_1, secondary_y=False)
fig.update_yaxes(title_text=column_2, secondary_y=True)
fig.show()
## Electricity generated from renewables
sns.set(style='whitegrid')
# Crear un gráfico de líneas para cada continente
plt.figure(figsize=(10, 6))
# Iterar a través de los nombres únicos de los continentes en df_all
for continente in df_all['continent'].unique():
df_continente = df_all[df_all['continent'] == continente]
# Crear el gráfico de líneas para el continente actual
sns.lineplot(data=df_continente, x='year', y='financial_flows_to_developing_countries(us$)', label=continente)
# Personalizar el diseño del gráfico
plt.xlabel('Año', fontweight='bold')
plt.ylabel('us$', fontweight='bold')
plt.title('financial_flows_to_developing_countries(us$) (2000 - 2020)', fontweight='bold')
plt.legend(title='Continente')
plt.xticks(rotation=45, ticks=df_all['year'].unique())
plt.show()
df_all.head()
entity | year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | density(P/km2) | landarea(km2) | continent | primary_energy_intensity | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | Albania | 2000 | 100.0 | 38.2 | 0.0 | 0.0 | 0.0 | 0.14 | 0.0 | 4.55 | 97.014930 | 9029.4375 | 4.13 | 3170.000000 | 0.0 | 6.946217 | 1126.683340 | 105 | 28748.0 | Europe | 4.13 |
1 | Albania | 2001 | 100.0 | 40.5 | 0.0 | 0.0 | 0.0 | 0.13 | 0.0 | 3.52 | 96.438354 | 8635.5320 | 3.89 | 3230.000000 | 0.0 | 8.293313 | 1281.659826 | 105 | 28748.0 | Europe | 3.89 |
2 | Albania | 2002 | 100.0 | 43.2 | 0.0 | 0.0 | 0.0 | 0.16 | 0.0 | 3.48 | 95.604390 | 9443.5550 | 4.10 | 3759.999990 | 0.0 | 4.536524 | 1425.124219 | 105 | 28748.0 | Europe | 4.10 |
3 | Albania | 2003 | 100.0 | 46.4 | 0.0 | 0.0 | 0.0 | 0.10 | 0.0 | 5.12 | 98.084300 | 10756.6120 | 3.80 | 4070.000172 | 0.0 | 5.528637 | 1846.120121 | 105 | 28748.0 | Europe | 3.80 |
4 | Albania | 2004 | 100.0 | 49.0 | 0.0 | 0.0 | 0.0 | 0.13 | 0.0 | 5.41 | 97.653430 | 11586.9510 | 3.96 | 4250.000000 | 0.0 | 5.514668 | 2373.581292 | 105 | 28748.0 | Europe | 3.96 |
df_all.isnull().sum().sort_values(ascending=False)
entity 0 primary_energy_consumption_per_capita(kwh/person) 0 continent 0 landarea(km2) 0 density(P/km2) 0 gdp_per_capita 0 gdp_growth 0 renewables(%equivalent_primary_energy) 0 CO2_emissions_value_by_country_kt 0 energy_intensity_level_of_primary_energy(mj/$2017pppgdp) 0 low_carbonelectricity(%electricity) 0 year 0 electricity_from_renewables(twh) 0 electricity_from_nuclear(twh) 0 electricity_from_fossil_fuels(twh) 0 renewable_energy_share_in_the_total_final_energy_consumption(%) 0 financial_flows_to_developing_countries(us$) 0 renewable_electricity_generating_capacity_per_capita 0 access_to_clean_fuels_for_cooking 0 access_to_electricity(%_of_population) 0 primary_energy_intensity 0 dtype: int64
df_all_cat=df_all.select_dtypes(include=['object'])
df_all_cat.head(5)
entity | density(P/km2) | continent | |
---|---|---|---|
0 | Albania | 105 | Europe |
1 | Albania | 105 | Europe |
2 | Albania | 105 | Europe |
3 | Albania | 105 | Europe |
4 | Albania | 105 | Europe |
df_all_num= df_all.select_dtypes(exclude=["object"])
df_all_num.head(5)
year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | landarea(km2) | primary_energy_intensity | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | 2000 | 100.0 | 38.2 | 0.0 | 0.0 | 0.0 | 0.14 | 0.0 | 4.55 | 97.014930 | 9029.4375 | 4.13 | 3170.000000 | 0.0 | 6.946217 | 1126.683340 | 28748.0 | 4.13 |
1 | 2001 | 100.0 | 40.5 | 0.0 | 0.0 | 0.0 | 0.13 | 0.0 | 3.52 | 96.438354 | 8635.5320 | 3.89 | 3230.000000 | 0.0 | 8.293313 | 1281.659826 | 28748.0 | 3.89 |
2 | 2002 | 100.0 | 43.2 | 0.0 | 0.0 | 0.0 | 0.16 | 0.0 | 3.48 | 95.604390 | 9443.5550 | 4.10 | 3759.999990 | 0.0 | 4.536524 | 1425.124219 | 28748.0 | 4.10 |
3 | 2003 | 100.0 | 46.4 | 0.0 | 0.0 | 0.0 | 0.10 | 0.0 | 5.12 | 98.084300 | 10756.6120 | 3.80 | 4070.000172 | 0.0 | 5.528637 | 1846.120121 | 28748.0 | 3.80 |
4 | 2004 | 100.0 | 49.0 | 0.0 | 0.0 | 0.0 | 0.13 | 0.0 | 5.41 | 97.653430 | 11586.9510 | 3.96 | 4250.000000 | 0.0 | 5.514668 | 2373.581292 | 28748.0 | 3.96 |
df_all_num.isnull().sum().sort_values(ascending=False)
year 0 access_to_electricity(%_of_population) 0 landarea(km2) 0 gdp_per_capita 0 gdp_growth 0 renewables(%equivalent_primary_energy) 0 CO2_emissions_value_by_country_kt 0 energy_intensity_level_of_primary_energy(mj/$2017pppgdp) 0 primary_energy_consumption_per_capita(kwh/person) 0 low_carbonelectricity(%electricity) 0 electricity_from_renewables(twh) 0 electricity_from_nuclear(twh) 0 electricity_from_fossil_fuels(twh) 0 renewable_energy_share_in_the_total_final_energy_consumption(%) 0 financial_flows_to_developing_countries(us$) 0 renewable_electricity_generating_capacity_per_capita 0 access_to_clean_fuels_for_cooking 0 primary_energy_intensity 0 dtype: int64
corr = df_all_num.corr()
ones = np.ones_like(corr, dtype="bool")
mask = np.triu(ones)
mask = mask[1:, :-1]
corr = corr.iloc[1:,:-1].copy()
fig, ax = plt.subplots(figsize=(30,20))
sns.heatmap(corr, mask=mask, annot=True, fmt=".2f", cmap="Greens", linewidths=.6, cbar_kws={"shrink":.9})
ax.xaxis.tick_bottom()
plt.title("Correlations heatmap for numerical data".upper(), fontdict={"fontsize": 18}, loc="left")
Text(0.0, 1.0, 'CORRELATIONS HEATMAP FOR NUMERICAL DATA')
#High correlation:.80 to 1
#Mean correlation:.60 to .80
#Low correlation: .40 to .60
corr = df_all_num.corr()
high_corr = corr[corr > 0.80]
high_corr = high_corr.unstack().dropna()
high_corr = high_corr[high_corr.index.get_level_values(0) != high_corr.index.get_level_values(1)]
for index, value in high_corr.items():
variable_1, variable_2 = index
correlation_value = value
print("Variables:", variable_1, "and", variable_2)
print("Correlation:", correlation_value)
print("------------------------")
Variables: electricity_from_fossil_fuels(twh) and electricity_from_renewables(twh) Correlation: 0.8438797621769125 ------------------------ Variables: electricity_from_fossil_fuels(twh) and CO2_emissions_value_by_country_kt Correlation: 0.9897914559622468 ------------------------ Variables: electricity_from_renewables(twh) and electricity_from_fossil_fuels(twh) Correlation: 0.8438797621769125 ------------------------ Variables: electricity_from_renewables(twh) and CO2_emissions_value_by_country_kt Correlation: 0.8566907489874316 ------------------------ Variables: energy_intensity_level_of_primary_energy(mj/$2017pppgdp) and primary_energy_intensity Correlation: 1.0 ------------------------ Variables: CO2_emissions_value_by_country_kt and electricity_from_fossil_fuels(twh) Correlation: 0.9897914559622468 ------------------------ Variables: CO2_emissions_value_by_country_kt and electricity_from_renewables(twh) Correlation: 0.8566907489874316 ------------------------ Variables: primary_energy_intensity and energy_intensity_level_of_primary_energy(mj/$2017pppgdp) Correlation: 1.0 ------------------------
electricity_from_fossil_fuels(twh) and electricity_from_renewables(twh) electricity_from_fossil_fuels(twh) and CO2_emissions_value_by_country_kt CO2_emissions_value_by_country_kt and electricity_from_renewables(twh) CO2_emissions_value_by_country_kt and electricity_from_fossil_fuels(twh)
Correlations Results
Correlación entre electricidad de combustibles fósiles y electricidad de energías renovables: La correlación positiva de 0.8439 entre "electricidad_from_fossil_fuels(twh)" y "electricity_from_renewables(twh)" sugiere que, en general, los países que producen más electricidad a partir de fuentes de energía fósil también tienden a producir más electricidad a partir de fuentes renovables. Esto puede ser un reflejo de la inversión en infraestructura energética en general en esos países, donde algunos pueden estar avanzando en ambas fuentes de energía.
Correlación entre electricidad de combustibles fósiles y emisiones de CO2: La alta correlación de 0.9898 entre "electricidad_from_fossil_fuels(twh)" y "CO2_emissions_value_by_country_kt" indica que hay una fuerte relación entre la producción de electricidad a partir de fuentes de energía fósil y las emisiones de dióxido de carbono (CO2). Esto es esperado, ya que la quema de combustibles fósiles es una de las principales fuentes de emisiones de CO2 en todo el mundo.
Correlación entre electricidad de energías renovables y emisiones de CO2: La correlación positiva de 0.8567 entre "electricity_from_renewables(twh)" y "CO2_emissions_value_by_country_kt" sugiere que, en general, los países que producen más electricidad a partir de fuentes renovables también tienden a tener emisiones de CO2 más bajas. Esto es consistente con la idea de que el uso de fuentes de energía renovable puede contribuir a una menor huella de carbono.
En resumen, estas correlaciones sugieren que existe una relación significativa entre la producción de electricidad a partir de diferentes fuentes (fósiles y renovables) y las emisiones de CO2 a nivel de país.
Una correlación negativa significa que a medida que aumenta el porcentaje de energía renovable en el consumo final de energía, disminuye el acceso a la electricidad en la población. Esto podría deberse a que en algunos países o regiones, donde la energía renovable representa una parte significativa del suministro de energía, la infraestructura eléctrica puede no estar tan desarrollada como en áreas con una mayor dependencia de fuentes de energía no renovable. También podría sugerir que en países donde el acceso a la electricidad es un desafío, se depende más de fuentes de energía no renovable tradicionales para generar electricidad.
Una correlación negativa indica que a medida que aumenta el uso de energía renovable en el consumo final de energía, disminuye el acceso a combustibles limpios para cocinar. Esto podría explicarse por el hecho de que en algunas áreas con un mayor uso de energía renovable, es posible que no se haya abordado adecuadamente el acceso a fuentes de combustible limpio para cocinar, lo que podría indicar desigualdades en el desarrollo de infraestructura energética. También podría reflejar que en áreas con menor acceso a la electricidad, es más probable que la población dependa de combustibles tradicionales no limpios, como la biomasa o el carbón, para cocinar. En resumen, estas correlaciones sugieren que existe una relación inversa entre el uso de energía renovable y el acceso a servicios energéticos esenciales, como la electricidad y los combustibles limpios para cocinar, en los datos que estás analizando. Sin embargo, es importante recordar que la correlación no implica causalidad y que estas relaciones pueden estar influenciadas por una serie de factores socioeconómicos y políticos específicos de cada país o región.
df_all_num.columns
Index(['year', 'access_to_electricity(%_of_population)', 'access_to_clean_fuels_for_cooking', 'renewable_electricity_generating_capacity_per_capita', 'financial_flows_to_developing_countries(us$)', 'renewable_energy_share_in_the_total_final_energy_consumption(%)', 'electricity_from_fossil_fuels(twh)', 'electricity_from_nuclear(twh)', 'electricity_from_renewables(twh)', 'low_carbonelectricity(%electricity)', 'primary_energy_consumption_per_capita(kwh/person)', 'energy_intensity_level_of_primary_energy(mj/$2017pppgdp)', 'CO2_emissions_value_by_country_kt', 'renewables(%equivalent_primary_energy)', 'gdp_growth', 'gdp_per_capita', 'landarea(km2)', 'primary_energy_intensity'], dtype='object')
#VIF
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, r2_score
from statsmodels.stats.outliers_influence import variance_inflation_factor
from statsmodels.tools.tools import add_constant
features = ['year', 'access_to_electricity(%_of_population)',
'access_to_clean_fuels_for_cooking',
'renewable_electricity_generating_capacity_per_capita',
'financial_flows_to_developing_countries(us$)',
'renewable_energy_share_in_the_total_final_energy_consumption(%)',
'electricity_from_fossil_fuels(twh)', 'electricity_from_nuclear(twh)',
'electricity_from_renewables(twh)',
'low_carbonelectricity(%electricity)',
'primary_energy_consumption_per_capita(kwh/person)',
'energy_intensity_level_of_primary_energy(mj/$2017pppgdp)',
'CO2_emissions_value_by_country_kt',
'renewables(%equivalent_primary_energy)', 'gdp_growth',
'gdp_per_capita', 'landarea(km2)']
def calculate_vif(df, features):
vif, tolerance, r2sq = {}, {}, {}
# all the features that you want to examine
for feature in features:
# extract all the other features you will regress against
X = [f for f in features if f != feature]
X, y = df[X], df[feature]
# extract r-squared from the fit
r2 = LinearRegression().fit(X, y).score(X, y)
# calculate tolerance
r2sq[feature] = r2
tolerance[feature] = 1 - r2
# calculate VIF
vif[feature] = 1/(tolerance[feature])
# return VIF DataFrame
return pd.DataFrame({'VIF': vif, 'Tolerance': tolerance, 'R2': r2sq})
calculate_vif(df_all_num, features)
VIF | Tolerance | R2 | |
---|---|---|---|
year | 1.190918 | 0.839689 | 0.160311 |
access_to_electricity(%_of_population) | 2.922642 | 0.342156 | 0.657844 |
access_to_clean_fuels_for_cooking | 3.884237 | 0.257451 | 0.742549 |
renewable_electricity_generating_capacity_per_capita | 1.452721 | 0.688363 | 0.311637 |
financial_flows_to_developing_countries(us$) | 1.097872 | 0.910853 | 0.089147 |
renewable_energy_share_in_the_total_final_energy_consumption(%) | 4.389289 | 0.227827 | 0.772173 |
electricity_from_fossil_fuels(twh) | 77.874914 | 0.012841 | 0.987159 |
electricity_from_nuclear(twh) | 2.793878 | 0.357925 | 0.642075 |
electricity_from_renewables(twh) | 6.215692 | 0.160883 | 0.839117 |
low_carbonelectricity(%electricity) | 2.382878 | 0.419661 | 0.580339 |
primary_energy_consumption_per_capita(kwh/person) | 2.272300 | 0.440083 | 0.559917 |
energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | 1.448907 | 0.690175 | 0.309825 |
CO2_emissions_value_by_country_kt | 72.790063 | 0.013738 | 0.986262 |
renewables(%equivalent_primary_energy) | 1.935471 | 0.516670 | 0.483330 |
gdp_growth | 1.147938 | 0.871127 | 0.128873 |
gdp_per_capita | 2.136868 | 0.467975 | 0.532025 |
landarea(km2) | 3.182479 | 0.314220 | 0.685780 |
Interpreting R square k - If $R^2_k$ equals zero, variable k is not correlated with any other independent variable. Usually, multicollinearity is a potential problem when $R^2_k$ is greater than 0.75 and, a serious problem when $R^2_k$ is greater than 0.9.
Interpretation of the variance inflation factor: If $VIF_k = 1$, variable k is not correlated with any other independent variable. Multicollinearity is a potential problem when VIFk is greater than 5 and, a serious problem when it is greater than 10.
Interpretation of tolerance A tolerance of less than 0.20 or 0.10 indicates a multicollinearity problem.
features = ['access_to_electricity(%_of_population)',
'renewable_electricity_generating_capacity_per_capita',
'financial_flows_to_developing_countries(us$)',
'electricity_from_nuclear(twh)',
'energy_intensity_level_of_primary_energy(mj/$2017pppgdp)',
'renewables(%equivalent_primary_energy)',
'gdp_per_capita']
def calculate_vif(df, features):
vif, tolerance, r2sq = {}, {}, {}
# all the features that you want to examine
for feature in features:
# extract all the other features you will regress against
X = [f for f in features if f != feature]
X, y = df[X], df[feature]
# extract r-squared from the fit
r2 = LinearRegression().fit(X, y).score(X, y)
# calculate tolerance
r2sq[feature] = r2
tolerance[feature] = 1 - r2
# calculate VIF
vif[feature] = 1/(tolerance[feature])
# return VIF DataFrame
return pd.DataFrame({'VIF': vif, 'Tolerance': tolerance, 'R2': r2sq})
calculate_vif(df_all_num, features)
VIF | Tolerance | R2 | |
---|---|---|---|
access_to_electricity(%_of_population) | 1.129653 | 0.885228 | 0.114772 |
renewable_electricity_generating_capacity_per_capita | 1.054005 | 0.948762 | 0.051238 |
financial_flows_to_developing_countries(us$) | 1.020117 | 0.980279 | 0.019721 |
electricity_from_nuclear(twh) | 1.050373 | 0.952043 | 0.047957 |
energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | 1.044196 | 0.957675 | 0.042325 |
renewables(%equivalent_primary_energy) | 1.056590 | 0.946441 | 0.053559 |
gdp_per_capita | 1.259492 | 0.793971 | 0.206029 |
#Savin features selection
df_num_l=df_all_num[['access_to_electricity(%_of_population)', 'renewable_electricity_generating_capacity_per_capita', 'financial_flows_to_developing_countries(us$)', 'electricity_from_nuclear(twh)', 'energy_intensity_level_of_primary_energy(mj/$2017pppgdp)', 'renewables(%equivalent_primary_energy)', 'gdp_per_capita', 'primary_energy_consumption_per_capita(kwh/person)']]
df_num_l= df_num_l.reset_index()
df_num_l = df_num_l.drop(columns=['index'])
df_num_l.head()
access_to_electricity(%_of_population) | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | electricity_from_nuclear(twh) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | renewables(%equivalent_primary_energy) | gdp_per_capita | primary_energy_consumption_per_capita(kwh/person) | |
---|---|---|---|---|---|---|---|---|
0 | 100.0 | 0.0 | 0.0 | 0.0 | 4.13 | 0.0 | 1126.683340 | 9029.4375 |
1 | 100.0 | 0.0 | 0.0 | 0.0 | 3.89 | 0.0 | 1281.659826 | 8635.5320 |
2 | 100.0 | 0.0 | 0.0 | 0.0 | 4.10 | 0.0 | 1425.124219 | 9443.5550 |
3 | 100.0 | 0.0 | 0.0 | 0.0 | 3.80 | 0.0 | 1846.120121 | 10756.6120 |
4 | 100.0 | 0.0 | 0.0 | 0.0 | 3.96 | 0.0 | 2373.581292 | 11586.9510 |
#X-y split
y=df_num_l['primary_energy_consumption_per_capita(kwh/person)']
X=df_num_l.drop(['primary_energy_consumption_per_capita(kwh/person)'],axis=1)
X.head()
access_to_electricity(%_of_population) | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | electricity_from_nuclear(twh) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | renewables(%equivalent_primary_energy) | gdp_per_capita | |
---|---|---|---|---|---|---|---|
0 | 100.0 | 0.0 | 0.0 | 0.0 | 4.13 | 0.0 | 1126.683340 |
1 | 100.0 | 0.0 | 0.0 | 0.0 | 3.89 | 0.0 | 1281.659826 |
2 | 100.0 | 0.0 | 0.0 | 0.0 | 4.10 | 0.0 | 1425.124219 |
3 | 100.0 | 0.0 | 0.0 | 0.0 | 3.80 | 0.0 | 1846.120121 |
4 | 100.0 | 0.0 | 0.0 | 0.0 | 3.96 | 0.0 | 2373.581292 |
#Encodiing categorical data
from sklearn.preprocessing import OneHotEncoder
encoder=OneHotEncoder(drop='first').fit(df_all_cat)
encoder=encoder.transform(df_all_cat).toarray()
encoder
array([[1., 0., 0., ..., 1., 0., 0.], [1., 0., 0., ..., 1., 0., 0.], [1., 0., 0., ..., 1., 0., 0.], ..., [0., 0., 0., ..., 0., 0., 1.], [0., 0., 0., ..., 0., 0., 1.], [0., 0., 0., ..., 0., 0., 1.]])
#Normalizing numerical data
from sklearn.preprocessing import StandardScaler
transformer=StandardScaler().fit(X)
X_norm=transformer.transform(X)
X_norm
array([[ 0.43021915, -0.45166585, -0.30391553, ..., -0.20611336, -0.86534058, -0.80663581], [ 0.43021915, -0.45166585, -0.30391553, ..., -0.27625502, -0.86534058, -0.79944867], [ 0.43021915, -0.45166585, -0.30391553, ..., -0.21488107, -0.86534058, -0.79279541], ..., [ 0.41561275, 3.62571671, -0.20282563, ..., -0.59773758, 1.26163592, 0.00851787], [ 0.42291595, 3.62152094, -0.20282563, ..., -0.60358272, 1.26163592, -0.03859265], [ 0.43021915, 3.612449 , -0.20282563, ..., -0.60358272, 1.26163592, -0.14291956]])
X=np.concatenate((X_norm ,encoder),axis=1)
X
array([[ 0.43021915, -0.45166585, -0.30391553, ..., 1. , 0. , 0. ], [ 0.43021915, -0.45166585, -0.30391553, ..., 1. , 0. , 0. ], [ 0.43021915, -0.45166585, -0.30391553, ..., 1. , 0. , 0. ], ..., [ 0.41561275, 3.62571671, -0.20282563, ..., 0. , 0. , 1. ], [ 0.42291595, 3.62152094, -0.20282563, ..., 0. , 0. , 1. ], [ 0.43021915, 3.612449 , -0.20282563, ..., 0. , 0. , 1. ]])
X_train,X_test,y_train,y_test=train_test_split(X,y,test_size=0.3,random_state=42)
X_train.shape
(1652, 218)
X_test.shape
(708, 218)
# Crear y ajustar el modelo de regresión lineal
regression_model = LinearRegression()
regression_model.fit(X_train, y_train)
LinearRegression()
# Realizar las predicciones en conjunto de entrenamiento y conjunto de prueba
y_train_pred = regression_model.predict(X_train)
y_test_pred = regression_model.predict(X_test)
# Calcular las métricas
r2_train = r2_score(y_train, y_train_pred)
mse_train = mean_squared_error(y_train, y_train_pred)
mae_train = mean_absolute_error(y_train, y_train_pred)
rmse_train = sqrt(mse_train)
r2_test = r2_score(y_test, y_test_pred)
mse_test = mean_squared_error(y_test, y_test_pred)
mae_test = mean_absolute_error(y_test, y_test_pred)
rmse_test = sqrt(mse_test)
# Imprimir las métricas
print("Métricas en conjunto de entrenamiento:")
print("R^2:", r2_train)
print("MSE:", mse_train)
print("MAE:", mae_train)
print("RMSE:", rmse_train)
print("------------------------------")
print("Métricas en conjunto de prueba:")
print("R^2:", r2_test)
print("MSE:", mse_test)
print("MAE:", mae_test)
print("RMSE:", rmse_test)
Métricas en conjunto de entrenamiento: R^2: 0.9758707859909201 MSE: 34073094.5035957 MAE: 3027.9388690496367 RMSE: 5837.216331745441 ------------------------------ Métricas en conjunto de prueba: R^2: 0.9738471974986119 MSE: 43458965.83241701 MAE: 3445.431845155367 RMSE: 6592.3414529601705
#problema de overfitting
df_dev = pd.concat([continent_data['Europe'], continent_data['North America']], axis=0)
df_dev.reset_index(drop=True, inplace=True)
df_dev.head(10)
entity | year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | density(P/km2) | landarea(km2) | continent | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | Albania | 2000 | 100.0 | 38.2 | 0.0 | 0.0 | 0.0 | 0.14 | 0.0 | 4.55 | 97.014930 | 9029.4375 | 4.13 | 3170.000000 | 0.0 | 6.946217 | 1126.683340 | 105 | 28748.0 | Europe |
1 | Albania | 2001 | 100.0 | 40.5 | 0.0 | 0.0 | 0.0 | 0.13 | 0.0 | 3.52 | 96.438354 | 8635.5320 | 3.89 | 3230.000000 | 0.0 | 8.293313 | 1281.659826 | 105 | 28748.0 | Europe |
2 | Albania | 2002 | 100.0 | 43.2 | 0.0 | 0.0 | 0.0 | 0.16 | 0.0 | 3.48 | 95.604390 | 9443.5550 | 4.10 | 3759.999990 | 0.0 | 4.536524 | 1425.124219 | 105 | 28748.0 | Europe |
3 | Albania | 2003 | 100.0 | 46.4 | 0.0 | 0.0 | 0.0 | 0.10 | 0.0 | 5.12 | 98.084300 | 10756.6120 | 3.80 | 4070.000172 | 0.0 | 5.528637 | 1846.120121 | 105 | 28748.0 | Europe |
4 | Albania | 2004 | 100.0 | 49.0 | 0.0 | 0.0 | 0.0 | 0.13 | 0.0 | 5.41 | 97.653430 | 11586.9510 | 3.96 | 4250.000000 | 0.0 | 5.514668 | 2373.581292 | 105 | 28748.0 | Europe |
5 | Albania | 2005 | 100.0 | 51.9 | 0.0 | 0.0 | 0.0 | 0.07 | 0.0 | 5.32 | 98.701300 | 11545.6160 | 3.75 | 4030.000210 | 0.0 | 5.526424 | 2673.786584 | 105 | 28748.0 | Europe |
6 | Albania | 2006 | 100.0 | 54.8 | 0.0 | 0.0 | 0.0 | 0.09 | 0.0 | 4.95 | 98.214280 | 10976.7280 | 3.46 | 4010.000229 | 0.0 | 5.902659 | 2972.742924 | 105 | 28748.0 | Europe |
7 | Albania | 2007 | 100.0 | 58.1 | 0.0 | 0.0 | 0.0 | 0.07 | 0.0 | 2.76 | 97.526505 | 9534.8740 | 3.12 | 4139.999866 | 0.0 | 5.983260 | 3595.038057 | 105 | 28748.0 | Europe |
8 | Albania | 2008 | 100.0 | 60.8 | 0.0 | 0.0 | 0.0 | 0.07 | 0.0 | 3.76 | 98.172325 | 10499.3280 | 3.01 | 4079.999924 | 0.0 | 7.500041 | 4370.539925 | 105 | 28748.0 | Europe |
9 | Albania | 2009 | 100.0 | 63.6 | 0.0 | 0.0 | 0.0 | 0.00 | 0.0 | 5.20 | 100.000000 | 11524.4280 | 2.98 | 4219.999790 | 0.0 | 3.354289 | 4114.134899 | 105 | 28748.0 | Europe |
df_dev.shape
(1330, 20)
df_dev_num= df_dev.select_dtypes(exclude=["object"])
df_dev_num.head(5)
year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | landarea(km2) | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | 2000 | 100.0 | 38.2 | 0.0 | 0.0 | 0.0 | 0.14 | 0.0 | 4.55 | 97.014930 | 9029.4375 | 4.13 | 3170.000000 | 0.0 | 6.946217 | 1126.683340 | 28748.0 |
1 | 2001 | 100.0 | 40.5 | 0.0 | 0.0 | 0.0 | 0.13 | 0.0 | 3.52 | 96.438354 | 8635.5320 | 3.89 | 3230.000000 | 0.0 | 8.293313 | 1281.659826 | 28748.0 |
2 | 2002 | 100.0 | 43.2 | 0.0 | 0.0 | 0.0 | 0.16 | 0.0 | 3.48 | 95.604390 | 9443.5550 | 4.10 | 3759.999990 | 0.0 | 4.536524 | 1425.124219 | 28748.0 |
3 | 2003 | 100.0 | 46.4 | 0.0 | 0.0 | 0.0 | 0.10 | 0.0 | 5.12 | 98.084300 | 10756.6120 | 3.80 | 4070.000172 | 0.0 | 5.528637 | 1846.120121 | 28748.0 |
4 | 2004 | 100.0 | 49.0 | 0.0 | 0.0 | 0.0 | 0.13 | 0.0 | 5.41 | 97.653430 | 11586.9510 | 3.96 | 4250.000000 | 0.0 | 5.514668 | 2373.581292 | 28748.0 |
df_dev_num.isnull().sum().sort_values(ascending=False)
year 0 low_carbonelectricity(%electricity) 0 gdp_per_capita 0 gdp_growth 0 renewables(%equivalent_primary_energy) 0 CO2_emissions_value_by_country_kt 0 energy_intensity_level_of_primary_energy(mj/$2017pppgdp) 0 primary_energy_consumption_per_capita(kwh/person) 0 electricity_from_renewables(twh) 0 access_to_electricity(%_of_population) 0 electricity_from_nuclear(twh) 0 electricity_from_fossil_fuels(twh) 0 renewable_energy_share_in_the_total_final_energy_consumption(%) 0 financial_flows_to_developing_countries(us$) 0 renewable_electricity_generating_capacity_per_capita 0 access_to_clean_fuels_for_cooking 0 landarea(km2) 0 dtype: int64
df_dev_cat= df_dev.select_dtypes(include=["object"])
df_dev_cat.head(5)
entity | density(P/km2) | continent | |
---|---|---|---|
0 | Albania | 105 | Europe |
1 | Albania | 105 | Europe |
2 | Albania | 105 | Europe |
3 | Albania | 105 | Europe |
4 | Albania | 105 | Europe |
corr = df_dev_num.corr()
high_corr = corr[corr > 0.80]
high_corr = high_corr.unstack().dropna()
high_corr = high_corr[high_corr.index.get_level_values(0) != high_corr.index.get_level_values(1)]
for index, value in high_corr.items():
variable_1, variable_2 = index
correlation_value = value
print("Variables:", variable_1, "and", variable_2)
print("Correlation:", correlation_value)
print("------------------------")
Variables: electricity_from_fossil_fuels(twh) and electricity_from_nuclear(twh) Correlation: 0.8730861592407058 ------------------------ Variables: electricity_from_fossil_fuels(twh) and CO2_emissions_value_by_country_kt Correlation: 0.9953098931166098 ------------------------ Variables: electricity_from_nuclear(twh) and electricity_from_fossil_fuels(twh) Correlation: 0.8730861592407058 ------------------------ Variables: electricity_from_nuclear(twh) and CO2_emissions_value_by_country_kt Correlation: 0.8924513724924181 ------------------------ Variables: electricity_from_renewables(twh) and landarea(km2) Correlation: 0.9022555182673455 ------------------------ Variables: CO2_emissions_value_by_country_kt and electricity_from_fossil_fuels(twh) Correlation: 0.9953098931166098 ------------------------ Variables: CO2_emissions_value_by_country_kt and electricity_from_nuclear(twh) Correlation: 0.8924513724924181 ------------------------ Variables: landarea(km2) and electricity_from_renewables(twh) Correlation: 0.9022555182673455 ------------------------
features = ['access_to_electricity(%_of_population)',
'renewable_electricity_generating_capacity_per_capita',
'financial_flows_to_developing_countries(us$)',
'electricity_from_nuclear(twh)',
'energy_intensity_level_of_primary_energy(mj/$2017pppgdp)',
'renewables(%equivalent_primary_energy)',
'gdp_per_capita']
def calculate_vif(df, features):
vif, tolerance, r2sq = {}, {}, {}
# all the features that you want to examine
for feature in features:
# extract all the other features you will regress against
X = [f for f in features if f != feature]
X, y = df[X], df[feature]
# extract r-squared from the fit
r2 = LinearRegression().fit(X, y).score(X, y)
# calculate tolerance
r2sq[feature] = r2
tolerance[feature] = 1 - r2
# calculate VIF
vif[feature] = 1/(tolerance[feature])
# return VIF DataFrame
return pd.DataFrame({'VIF': vif, 'Tolerance': tolerance, 'R2': r2sq})
calculate_vif(df_dev_num, features)
VIF | Tolerance | R2 | |
---|---|---|---|
access_to_electricity(%_of_population) | 1.165409 | 0.858068 | 0.141932 |
renewable_electricity_generating_capacity_per_capita | 1.166703 | 0.857116 | 0.142884 |
financial_flows_to_developing_countries(us$) | 1.164534 | 0.858712 | 0.141288 |
electricity_from_nuclear(twh) | 1.059800 | 0.943574 | 0.056426 |
energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | 1.068266 | 0.936096 | 0.063904 |
renewables(%equivalent_primary_energy) | 1.132474 | 0.883022 | 0.116978 |
gdp_per_capita | 1.273213 | 0.785415 | 0.214585 |
#Savin features selection
df_dev_numl=df_dev_num[['access_to_electricity(%_of_population)', 'renewable_electricity_generating_capacity_per_capita', 'financial_flows_to_developing_countries(us$)', 'electricity_from_nuclear(twh)', 'energy_intensity_level_of_primary_energy(mj/$2017pppgdp)', 'renewables(%equivalent_primary_energy)', 'gdp_per_capita', 'primary_energy_consumption_per_capita(kwh/person)']]
df_dev_numl= df_dev_numl.reset_index()
df_dev_numl = df_dev_numl.drop(columns=['index'])
df_dev_numl.head()
access_to_electricity(%_of_population) | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | electricity_from_nuclear(twh) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | renewables(%equivalent_primary_energy) | gdp_per_capita | primary_energy_consumption_per_capita(kwh/person) | |
---|---|---|---|---|---|---|---|---|
0 | 100.0 | 0.0 | 0.0 | 0.0 | 4.13 | 0.0 | 1126.683340 | 9029.4375 |
1 | 100.0 | 0.0 | 0.0 | 0.0 | 3.89 | 0.0 | 1281.659826 | 8635.5320 |
2 | 100.0 | 0.0 | 0.0 | 0.0 | 4.10 | 0.0 | 1425.124219 | 9443.5550 |
3 | 100.0 | 0.0 | 0.0 | 0.0 | 3.80 | 0.0 | 1846.120121 | 10756.6120 |
4 | 100.0 | 0.0 | 0.0 | 0.0 | 3.96 | 0.0 | 2373.581292 | 11586.9510 |
#X-y split
y=df_dev_numl['primary_energy_consumption_per_capita(kwh/person)']
X_l=df_dev_numl.drop(['primary_energy_consumption_per_capita(kwh/person)'],axis=1)
X_l.head()
access_to_electricity(%_of_population) | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | electricity_from_nuclear(twh) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | renewables(%equivalent_primary_energy) | gdp_per_capita | |
---|---|---|---|---|---|---|---|
0 | 100.0 | 0.0 | 0.0 | 0.0 | 4.13 | 0.0 | 1126.683340 |
1 | 100.0 | 0.0 | 0.0 | 0.0 | 3.89 | 0.0 | 1281.659826 |
2 | 100.0 | 0.0 | 0.0 | 0.0 | 4.10 | 0.0 | 1425.124219 |
3 | 100.0 | 0.0 | 0.0 | 0.0 | 3.80 | 0.0 | 1846.120121 |
4 | 100.0 | 0.0 | 0.0 | 0.0 | 3.96 | 0.0 | 2373.581292 |
#Encodiing categorical data
from sklearn.preprocessing import OneHotEncoder
encoder=OneHotEncoder(drop='first').fit(df_dev_cat)
encoder=encoder.transform(df_dev_cat).toarray()
encoder
array([[0., 0., 0., ..., 0., 0., 0.], [0., 0., 0., ..., 0., 0., 0.], [0., 0., 0., ..., 0., 0., 0.], ..., [0., 0., 0., ..., 0., 0., 1.], [0., 0., 0., ..., 0., 0., 1.], [0., 0., 0., ..., 0., 0., 1.]])
#Normalizing numerical data
from sklearn.preprocessing import StandardScaler
transformer=StandardScaler().fit(X_l)
X_norml=transformer.transform(X_l)
X_norml
array([[ 0.3467588 , -0.45622589, -0.25203343, ..., -0.10885733, -0.92886499, -0.98138141], [ 0.3467588 , -0.45622589, -0.25203343, ..., -0.18643203, -0.92886499, -0.97495576], [ 0.3467588 , -0.45622589, -0.25203343, ..., -0.11855417, -0.92886499, -0.96900743], ..., [ 0.3467588 , -0.43407441, -0.18215014, ..., 0.04629207, -0.36627213, 1.5866868 ], [ 0.3467588 , -0.43407441, -0.18215014, ..., 0.01396928, -0.34428762, 1.67852992], [ 0.3467588 , -0.43407441, -0.18215014, ..., 0.01396928, -0.26791699, 1.60862129]])
X_norml=np.concatenate((X_norml ,encoder),axis=1)
X_norml
array([[ 0.3467588 , -0.45622589, -0.25203343, ..., 0. , 0. , 0. ], [ 0.3467588 , -0.45622589, -0.25203343, ..., 0. , 0. , 0. ], [ 0.3467588 , -0.45622589, -0.25203343, ..., 0. , 0. , 0. ], ..., [ 0.3467588 , -0.43407441, -0.18215014, ..., 0. , 0. , 1. ], [ 0.3467588 , -0.43407441, -0.18215014, ..., 0. , 0. , 1. ], [ 0.3467588 , -0.43407441, -0.18215014, ..., 0. , 0. , 1. ]])
X_train,X_test,y_train,y_test=train_test_split(X_norml,y,test_size=0.3,random_state=42)
X_train.shape
(931, 129)
X_test.shape
(399, 129)
# Crear y ajustar el modelo de regresión lineal
regression_model = LinearRegression()
regression_model.fit(X_train, y_train)
LinearRegression()
# Realizar las predicciones en conjunto de entrenamiento y conjunto de prueba
y_train_pred = regression_model.predict(X_train)
y_test_pred = regression_model.predict(X_test)
# Calcular las métricas
r2_train = r2_score(y_train, y_train_pred)
mse_train = mean_squared_error(y_train, y_train_pred)
mae_train = mean_absolute_error(y_train, y_train_pred)
rmse_train = sqrt(mse_train)
r2_test = r2_score(y_test, y_test_pred)
mse_test = mean_squared_error(y_test, y_test_pred)
mae_test = mean_absolute_error(y_test, y_test_pred)
rmse_test = sqrt(mse_test)
# Imprimir las métricas
print("Métricas en conjunto de entrenamiento:")
print("R^2:", r2_train)
print("MSE:", mse_train)
print("MAE:", mae_train)
print("RMSE:", rmse_train)
print("------------------------------")
print("Métricas en conjunto de prueba:")
print("R^2:", r2_test)
print("MSE:", mse_test)
print("MAE:", mae_test)
print("RMSE:", rmse_test)
Métricas en conjunto de entrenamiento: R^2: 0.9688619837329396 MSE: 30379143.381679874 MAE: 3127.1883642749735 RMSE: 5511.727803663736 ------------------------------ Métricas en conjunto de prueba: R^2: 0.9625653206802071 MSE: 40810421.401613295 MAE: 3332.389559398496 RMSE: 6388.303483837732
df_sub = pd.concat([continent_data['South America'], continent_data['Asia']], axis=0)
df_sub.reset_index(drop=True, inplace=True)
df_sub.head(10)
entity | year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | density(P/km2) | landarea(km2) | continent | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | Argentina | 2000 | 95.783290 | 95.00 | 235.62 | 0.0 | 9.85 | 50.37 | 5.99 | 28.89 | 40.914955 | 18849.810 | 3.69 | 132270.0000 | 12.283858 | -0.788999 | 7708.099115 | 17 | 2780400.0 | South America |
1 | Argentina | 2001 | 95.511060 | 95.60 | 234.09 | 3890000.0 | 11.51 | 42.91 | 6.54 | 37.04 | 50.387330 | 18210.120 | 3.59 | 125260.0000 | 15.937488 | -4.408840 | 7208.373114 | 17 | 2780400.0 | South America |
2 | Argentina | 2002 | 96.228870 | 96.10 | 236.78 | 3890000.0 | 11.62 | 39.70 | 5.39 | 36.06 | 51.078250 | 17275.256 | 3.99 | 117470.0012 | 16.249306 | -10.894485 | 2593.404563 | 17 | 2780400.0 | South America |
3 | Argentina | 2003 | 96.442635 | 96.45 | 234.82 | 50000.0 | 10.82 | 46.73 | 7.03 | 34.38 | 46.982075 | 18326.521 | 3.97 | 127660.0037 | 14.408259 | 8.837041 | 3349.806300 | 17 | 2780400.0 | South America |
4 | Argentina | 2004 | 96.653500 | 97.20 | 236.15 | 40960000.0 | 9.29 | 57.35 | 7.31 | 31.03 | 40.066883 | 19261.457 | 4.04 | 141380.0049 | 12.185576 | 9.029573 | 4277.721573 | 17 | 2780400.0 | South America |
5 | Argentina | 2005 | 96.863846 | 97.50 | 234.67 | 280000.0 | 9.64 | 59.89 | 6.37 | 34.60 | 40.620663 | 20232.844 | 3.68 | 145990.0055 | 12.788524 | 8.851660 | 5109.852245 | 17 | 2780400.0 | South America |
6 | Argentina | 2006 | 97.076060 | 98.10 | 232.56 | 440000.0 | 9.18 | 65.58 | 7.15 | 38.67 | 41.131058 | 21038.723 | 3.58 | 154899.9939 | 13.449462 | 8.047152 | 5919.012338 | 17 | 2780400.0 | South America |
7 | Argentina | 2007 | 97.292790 | 98.30 | 236.60 | 360000.0 | 7.72 | 72.95 | 6.72 | 31.18 | 34.190350 | 21708.370 | 3.44 | 162809.9976 | 10.599154 | 9.007651 | 7245.446857 | 17 | 2780400.0 | South America |
8 | Argentina | 2008 | 97.516754 | 98.60 | 234.79 | 22940000.0 | 7.65 | 76.62 | 6.85 | 31.28 | 33.228760 | 21780.380 | 3.42 | 167229.9957 | 10.437064 | 4.057233 | 9020.873323 | 17 | 2780400.0 | South America |
9 | Argentina | 2009 | 97.750656 | 98.90 | 241.83 | 39080000.0 | 8.59 | 74.01 | 7.59 | 34.70 | 36.362854 | 20954.271 | 3.35 | 156570.0073 | 11.783506 | -5.918525 | 8225.137583 | 17 | 2780400.0 | South America |
df_sub.shape
(1030, 20)
df_sub_num= df_sub.select_dtypes(exclude=["object"])
df_sub_num.head(5)
year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | landarea(km2) | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | 2000 | 95.783290 | 95.00 | 235.62 | 0.0 | 9.85 | 50.37 | 5.99 | 28.89 | 40.914955 | 18849.810 | 3.69 | 132270.0000 | 12.283858 | -0.788999 | 7708.099115 | 2780400.0 |
1 | 2001 | 95.511060 | 95.60 | 234.09 | 3890000.0 | 11.51 | 42.91 | 6.54 | 37.04 | 50.387330 | 18210.120 | 3.59 | 125260.0000 | 15.937488 | -4.408840 | 7208.373114 | 2780400.0 |
2 | 2002 | 96.228870 | 96.10 | 236.78 | 3890000.0 | 11.62 | 39.70 | 5.39 | 36.06 | 51.078250 | 17275.256 | 3.99 | 117470.0012 | 16.249306 | -10.894485 | 2593.404563 | 2780400.0 |
3 | 2003 | 96.442635 | 96.45 | 234.82 | 50000.0 | 10.82 | 46.73 | 7.03 | 34.38 | 46.982075 | 18326.521 | 3.97 | 127660.0037 | 14.408259 | 8.837041 | 3349.806300 | 2780400.0 |
4 | 2004 | 96.653500 | 97.20 | 236.15 | 40960000.0 | 9.29 | 57.35 | 7.31 | 31.03 | 40.066883 | 19261.457 | 4.04 | 141380.0049 | 12.185576 | 9.029573 | 4277.721573 | 2780400.0 |
df_sub_cat= df_sub.select_dtypes(include=["object"])
df_sub_cat.head(5)
entity | density(P/km2) | continent | |
---|---|---|---|
0 | Argentina | 17 | South America |
1 | Argentina | 17 | South America |
2 | Argentina | 17 | South America |
3 | Argentina | 17 | South America |
4 | Argentina | 17 | South America |
corr = df_sub_num.corr()
high_corr = corr[corr > 0.80]
high_corr = high_corr.unstack().dropna()
high_corr = high_corr[high_corr.index.get_level_values(0) != high_corr.index.get_level_values(1)]
for index, value in high_corr.items():
variable_1, variable_2 = index
correlation_value = value
print("Variables:", variable_1, "and", variable_2)
print("Correlation:", correlation_value)
print("------------------------")
Variables: electricity_from_fossil_fuels(twh) and electricity_from_renewables(twh) Correlation: 0.9034233282584758 ------------------------ Variables: electricity_from_fossil_fuels(twh) and CO2_emissions_value_by_country_kt Correlation: 0.9916684463061797 ------------------------ Variables: electricity_from_renewables(twh) and electricity_from_fossil_fuels(twh) Correlation: 0.9034233282584758 ------------------------ Variables: electricity_from_renewables(twh) and CO2_emissions_value_by_country_kt Correlation: 0.8953382454246934 ------------------------ Variables: primary_energy_consumption_per_capita(kwh/person) and gdp_per_capita Correlation: 0.8020925986431613 ------------------------ Variables: CO2_emissions_value_by_country_kt and electricity_from_fossil_fuels(twh) Correlation: 0.9916684463061797 ------------------------ Variables: CO2_emissions_value_by_country_kt and electricity_from_renewables(twh) Correlation: 0.8953382454246934 ------------------------ Variables: gdp_per_capita and primary_energy_consumption_per_capita(kwh/person) Correlation: 0.8020925986431613 ------------------------
features = ['access_to_electricity(%_of_population)',
'renewable_electricity_generating_capacity_per_capita',
'financial_flows_to_developing_countries(us$)',
'electricity_from_nuclear(twh)',
'energy_intensity_level_of_primary_energy(mj/$2017pppgdp)',
'renewables(%equivalent_primary_energy)',
'gdp_per_capita']
def calculate_vif(df, features):
vif, tolerance, r2sq = {}, {}, {}
# all the features that you want to examine
for feature in features:
# extract all the other features you will regress against
X = [f for f in features if f != feature]
X, y = df[X], df[feature]
# extract r-squared from the fit
r2 = LinearRegression().fit(X, y).score(X, y)
# calculate tolerance
r2sq[feature] = r2
tolerance[feature] = 1 - r2
# calculate VIF
vif[feature] = 1/(tolerance[feature])
# return VIF DataFrame
return pd.DataFrame({'VIF': vif, 'Tolerance': tolerance, 'R2': r2sq})
calculate_vif(df_sub_num, features)
VIF | Tolerance | R2 | |
---|---|---|---|
access_to_electricity(%_of_population) | 1.238804 | 0.807230 | 0.192770 |
renewable_electricity_generating_capacity_per_capita | 1.119031 | 0.893631 | 0.106369 |
financial_flows_to_developing_countries(us$) | 1.020908 | 0.979521 | 0.020479 |
electricity_from_nuclear(twh) | 1.036540 | 0.964748 | 0.035252 |
energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | 1.136006 | 0.880277 | 0.119723 |
renewables(%equivalent_primary_energy) | 1.319612 | 0.757798 | 0.242202 |
gdp_per_capita | 1.326972 | 0.753595 | 0.246405 |
df_sub_numl=df_sub_num[['access_to_electricity(%_of_population)', 'renewable_electricity_generating_capacity_per_capita', 'financial_flows_to_developing_countries(us$)', 'electricity_from_nuclear(twh)', 'energy_intensity_level_of_primary_energy(mj/$2017pppgdp)', 'renewables(%equivalent_primary_energy)', 'gdp_per_capita', 'primary_energy_consumption_per_capita(kwh/person)']]
df_sub_numl= df_sub_numl.reset_index()
df_sub_numl = df_sub_numl.drop(columns=['index'])
df_sub_numl.head()
access_to_electricity(%_of_population) | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | electricity_from_nuclear(twh) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | renewables(%equivalent_primary_energy) | gdp_per_capita | primary_energy_consumption_per_capita(kwh/person) | |
---|---|---|---|---|---|---|---|---|
0 | 95.783290 | 235.62 | 0.0 | 5.99 | 3.69 | 12.283858 | 7708.099115 | 18849.810 |
1 | 95.511060 | 234.09 | 3890000.0 | 6.54 | 3.59 | 15.937488 | 7208.373114 | 18210.120 |
2 | 96.228870 | 236.78 | 3890000.0 | 5.39 | 3.99 | 16.249306 | 2593.404563 | 17275.256 |
3 | 96.442635 | 234.82 | 50000.0 | 7.03 | 3.97 | 14.408259 | 3349.806300 | 18326.521 |
4 | 96.653500 | 236.15 | 40960000.0 | 7.31 | 4.04 | 12.185576 | 4277.721573 | 19261.457 |
#X-y split
y=df_sub_numl['primary_energy_consumption_per_capita(kwh/person)']
X_sub_l=df_sub_numl.drop(['primary_energy_consumption_per_capita(kwh/person)'],axis=1)
X_sub_l.head()
access_to_electricity(%_of_population) | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | electricity_from_nuclear(twh) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | renewables(%equivalent_primary_energy) | gdp_per_capita | |
---|---|---|---|---|---|---|---|
0 | 95.783290 | 235.62 | 0.0 | 5.99 | 3.69 | 12.283858 | 7708.099115 |
1 | 95.511060 | 234.09 | 3890000.0 | 6.54 | 3.59 | 15.937488 | 7208.373114 |
2 | 96.228870 | 236.78 | 3890000.0 | 5.39 | 3.99 | 16.249306 | 2593.404563 |
3 | 96.442635 | 234.82 | 50000.0 | 7.03 | 3.97 | 14.408259 | 3349.806300 |
4 | 96.653500 | 236.15 | 40960000.0 | 7.31 | 4.04 | 12.185576 | 4277.721573 |
#Encodiing categorical data
from sklearn.preprocessing import OneHotEncoder
encoder=OneHotEncoder(drop='first').fit(df_sub_cat)
encoder_sub=encoder.transform(df_sub_cat).toarray()
encoder_sub
array([[1., 0., 0., ..., 0., 0., 1.], [1., 0., 0., ..., 0., 0., 1.], [1., 0., 0., ..., 0., 0., 1.], ..., [0., 0., 0., ..., 0., 0., 0.], [0., 0., 0., ..., 0., 0., 0.], [0., 0., 0., ..., 0., 0., 0.]])
#Normalizing numerical data
from sklearn.preprocessing import StandardScaler
transformer=StandardScaler().fit(X_sub_l)
X_sub_norml=transformer.transform(X_sub_l)
X_sub_norml
array([[ 0.29795951, 0.05245026, -0.43165674, ..., -0.43224796, 0.23536149, -0.19261716], [ 0.28218911, 0.04823046, -0.41660427, ..., -0.45891294, 0.54950488, -0.22815805], [ 0.32377215, 0.05564959, -0.41660427, ..., -0.35225303, 0.57631536, -0.55637807], ..., [-1.65912085, -0.57321177, -0.21600794, ..., -0.86945855, -0.60385232, -0.61631205], [-1.03630645, -0.57376338, -0.4270133 , ..., -0.86945855, -0.60385232, -0.61631205], [-0.97797897, -0.5740116 , -0.4270133 , ..., -0.86945855, -0.60385232, -0.61631205]])
X_sub_norml=np.concatenate((X_sub_norml ,encoder_sub),axis=1)
X_sub_norml
array([[ 0.29795951, 0.05245026, -0.43165674, ..., 0. , 0. , 1. ], [ 0.28218911, 0.04823046, -0.41660427, ..., 0. , 0. , 1. ], [ 0.32377215, 0.05564959, -0.41660427, ..., 0. , 0. , 1. ], ..., [-1.65912085, -0.57321177, -0.21600794, ..., 0. , 0. , 0. ], [-1.03630645, -0.57376338, -0.4270133 , ..., 0. , 0. , 0. ], [-0.97797897, -0.5740116 , -0.4270133 , ..., 0. , 0. , 0. ]])
X_train,X_test,y_train,y_test=train_test_split(X_sub_norml,y,test_size=0.3,random_state=42)
X_train.shape
(721, 195)
X_test.shape
(309, 195)
# Crear y ajustar el modelo de regresión lineal
regression_model = LinearRegression()
regression_model.fit(X_train, y_train)
LinearRegression()
# Realizar las predicciones en conjunto de entrenamiento y conjunto de prueba
y_train_pred = regression_model.predict(X_train)
y_test_pred = regression_model.predict(X_test)
# Calcular las métricas
r2_train = r2_score(y_train, y_train_pred)
mse_train = mean_squared_error(y_train, y_train_pred)
mae_train = mean_absolute_error(y_train, y_train_pred)
rmse_train = sqrt(mse_train)
r2_test = r2_score(y_test, y_test_pred)
mse_test = mean_squared_error(y_test, y_test_pred)
mae_test = mean_absolute_error(y_test, y_test_pred)
rmse_test = sqrt(mse_test)
# Imprimir las métricas
print("Métricas en conjunto de entrenamiento:")
print("R^2:", r2_train)
print("MSE:", mse_train)
print("MAE:", mae_train)
print("RMSE:", rmse_train)
print("------------------------------")
print("Métricas en conjunto de prueba:")
print("R^2:", r2_test)
print("MSE:", mse_test)
print("MAE:", mae_test)
print("RMSE:", rmse_test)
Métricas en conjunto de entrenamiento: R^2: 0.9847793873800021 MSE: 30835007.348088704 MAE: 2763.2170265187237 RMSE: 5552.927817655179 ------------------------------ Métricas en conjunto de prueba: R^2: 0.9823556780123545 MSE: 39464487.59206792 MAE: 2960.9102841423955 RMSE: 6282.076694220465
df_all.head()
entity | year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | density(P/km2) | landarea(km2) | continent | primary_energy_intensity | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | Albania | 2000 | 100.0 | 38.2 | 0.0 | 0.0 | 0.0 | 0.14 | 0.0 | 4.55 | 97.014930 | 9029.4375 | 4.13 | 3170.000000 | 0.0 | 6.946217 | 1126.683340 | 105 | 28748.0 | Europe | 4.13 |
1 | Albania | 2001 | 100.0 | 40.5 | 0.0 | 0.0 | 0.0 | 0.13 | 0.0 | 3.52 | 96.438354 | 8635.5320 | 3.89 | 3230.000000 | 0.0 | 8.293313 | 1281.659826 | 105 | 28748.0 | Europe | 3.89 |
2 | Albania | 2002 | 100.0 | 43.2 | 0.0 | 0.0 | 0.0 | 0.16 | 0.0 | 3.48 | 95.604390 | 9443.5550 | 4.10 | 3759.999990 | 0.0 | 4.536524 | 1425.124219 | 105 | 28748.0 | Europe | 4.10 |
3 | Albania | 2003 | 100.0 | 46.4 | 0.0 | 0.0 | 0.0 | 0.10 | 0.0 | 5.12 | 98.084300 | 10756.6120 | 3.80 | 4070.000172 | 0.0 | 5.528637 | 1846.120121 | 105 | 28748.0 | Europe | 3.80 |
4 | Albania | 2004 | 100.0 | 49.0 | 0.0 | 0.0 | 0.0 | 0.13 | 0.0 | 5.41 | 97.653430 | 11586.9510 | 3.96 | 4250.000000 | 0.0 | 5.514668 | 2373.581292 | 105 | 28748.0 | Europe | 3.96 |
columnas_delete = ['density(P/km2)']
df_all = df_all.drop(columns=columnas_delete)
df_all.head(5)
entity | year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | landarea(km2) | continent | primary_energy_intensity | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | Albania | 2000 | 100.0 | 38.2 | 0.0 | 0.0 | 0.0 | 0.14 | 0.0 | 4.55 | 97.014930 | 9029.4375 | 4.13 | 3170.000000 | 0.0 | 6.946217 | 1126.683340 | 28748.0 | Europe | 4.13 |
1 | Albania | 2001 | 100.0 | 40.5 | 0.0 | 0.0 | 0.0 | 0.13 | 0.0 | 3.52 | 96.438354 | 8635.5320 | 3.89 | 3230.000000 | 0.0 | 8.293313 | 1281.659826 | 28748.0 | Europe | 3.89 |
2 | Albania | 2002 | 100.0 | 43.2 | 0.0 | 0.0 | 0.0 | 0.16 | 0.0 | 3.48 | 95.604390 | 9443.5550 | 4.10 | 3759.999990 | 0.0 | 4.536524 | 1425.124219 | 28748.0 | Europe | 4.10 |
3 | Albania | 2003 | 100.0 | 46.4 | 0.0 | 0.0 | 0.0 | 0.10 | 0.0 | 5.12 | 98.084300 | 10756.6120 | 3.80 | 4070.000172 | 0.0 | 5.528637 | 1846.120121 | 28748.0 | Europe | 3.80 |
4 | Albania | 2004 | 100.0 | 49.0 | 0.0 | 0.0 | 0.0 | 0.13 | 0.0 | 5.41 | 97.653430 | 11586.9510 | 3.96 | 4250.000000 | 0.0 | 5.514668 | 2373.581292 | 28748.0 | Europe | 3.96 |
df_all.describe()
year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | landarea(km2) | primary_energy_intensity | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
count | 2360.000000 | 2360.000000 | 2360.000000 | 2360.000000 | 2.360000e+03 | 2360.000000 | 2360.000000 | 2360.000000 | 2360.000000 | 2360.000000 | 2360.000000 | 2360.000000 | 2.360000e+03 | 2360.000000 | 2360.000000 | 2360.000000 | 2.360000e+03 | 2360.000000 |
mean | 2010.037288 | 94.109168 | 82.161801 | 119.489136 | 5.498685e+07 | 21.078839 | 101.782258 | 20.480377 | 35.351318 | 36.847134 | 35240.863611 | 4.835247 | 2.221648e+05 | 12.508003 | 2.774079 | 18520.189212 | 6.533489e+05 | 4.835247 |
std | 6.050680 | 13.695534 | 25.882369 | 264.608138 | 1.809664e+08 | 21.688226 | 427.242001 | 88.464947 | 127.965061 | 32.979542 | 38618.610111 | 3.422373 | 9.288601e+05 | 14.457491 | 5.145673 | 21567.592043 | 1.798739e+06 | 3.422373 |
min | 2000.000000 | 1.613591 | 2.600000 | 0.000000 | 0.000000e+00 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 0.000000 | 204.231250 | 0.110000 | 0.000000e+00 | 0.000000 | -36.658153 | 0.000000 | 5.400000e+01 | 0.110000 |
25% | 2005.000000 | 96.671796 | 74.812500 | 0.000000 | 0.000000e+00 | 3.695000 | 1.600000 | 0.000000 | 0.160000 | 5.855906 | 10987.687750 | 2.977500 | 6.517500e+03 | 0.907626 | 0.835889 | 3626.165399 | 2.571300e+04 | 2.977500 |
50% | 2010.000000 | 100.000000 | 95.600000 | 9.160000 | 4.600000e+05 | 14.200000 | 10.030000 | 0.000000 | 3.855000 | 30.333500 | 22499.266500 | 3.920000 | 2.720000e+04 | 7.163172 | 2.983055 | 8838.548931 | 9.302800e+04 | 3.920000 |
75% | 2015.000000 | 100.000000 | 100.000000 | 111.530000 | 1.955000e+07 | 31.605000 | 45.910000 | 1.907500 | 18.180000 | 62.339205 | 43192.171250 | 5.512500 | 9.819750e+04 | 19.903378 | 5.437322 | 28594.747228 | 3.779440e+05 | 5.512500 |
max | 2020.000000 | 100.000000 | 100.000000 | 3060.190000 | 3.386850e+09 | 93.460000 | 5184.130000 | 809.410000 | 2184.940000 | 100.000010 | 262585.700000 | 32.570000 | 1.070722e+07 | 86.836586 | 53.381794 | 123514.196700 | 9.984670e+06 | 32.570000 |
df_all_num= df_all.select_dtypes(exclude=["object"])
df_all_num.head(5)
year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | landarea(km2) | primary_energy_intensity | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | 2000 | 100.0 | 38.2 | 0.0 | 0.0 | 0.0 | 0.14 | 0.0 | 4.55 | 97.014930 | 9029.4375 | 4.13 | 3170.000000 | 0.0 | 6.946217 | 1126.683340 | 28748.0 | 4.13 |
1 | 2001 | 100.0 | 40.5 | 0.0 | 0.0 | 0.0 | 0.13 | 0.0 | 3.52 | 96.438354 | 8635.5320 | 3.89 | 3230.000000 | 0.0 | 8.293313 | 1281.659826 | 28748.0 | 3.89 |
2 | 2002 | 100.0 | 43.2 | 0.0 | 0.0 | 0.0 | 0.16 | 0.0 | 3.48 | 95.604390 | 9443.5550 | 4.10 | 3759.999990 | 0.0 | 4.536524 | 1425.124219 | 28748.0 | 4.10 |
3 | 2003 | 100.0 | 46.4 | 0.0 | 0.0 | 0.0 | 0.10 | 0.0 | 5.12 | 98.084300 | 10756.6120 | 3.80 | 4070.000172 | 0.0 | 5.528637 | 1846.120121 | 28748.0 | 3.80 |
4 | 2004 | 100.0 | 49.0 | 0.0 | 0.0 | 0.0 | 0.13 | 0.0 | 5.41 | 97.653430 | 11586.9510 | 3.96 | 4250.000000 | 0.0 | 5.514668 | 2373.581292 | 28748.0 | 3.96 |
df_all_cat=df_all.select_dtypes(include=['object'])
df_all_cat.head(5)
entity | continent | |
---|---|---|
0 | Albania | Europe |
1 | Albania | Europe |
2 | Albania | Europe |
3 | Albania | Europe |
4 | Albania | Europe |
scaler = StandardScaler()
df_all_num_normalized = scaler.fit_transform(df_all_num)
df_all_num_normalized
array([[-1.65922091, 0.43021915, -1.69888305, ..., -0.80663581, -0.34731743, -0.20611336], [-1.49391521, 0.43021915, -1.61000064, ..., -0.79944867, -0.34731743, -0.27625502], [-1.32860951, 0.43021915, -1.50566041, ..., -0.79279541, -0.34731743, -0.21488107], ..., [ 1.31628163, 0.41561275, 0.68934879, ..., 0.00851787, -0.26531649, -0.59773758], [ 1.48158733, 0.42291595, 0.68934879, ..., -0.03859265, -0.26531649, -0.60358272], [ 1.64689303, 0.43021915, 0.68934879, ..., -0.14291956, -0.26531649, -0.60358272]])
df_all_num_normalized= pd.DataFrame(df_all_num_normalized, columns= df_all_num.columns)
df_all_num_normalized.head()
year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | landarea(km2) | primary_energy_intensity | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | -1.659221 | 0.430219 | -1.698883 | -0.451666 | -0.303916 | -0.972108 | -0.237954 | -0.231557 | -0.240752 | 1.824784 | -0.678869 | -0.206113 | -0.235817 | -0.865341 | 0.810977 | -0.806636 | -0.347317 | -0.206113 |
1 | -1.493915 | 0.430219 | -1.610001 | -0.451666 | -0.303916 | -0.972108 | -0.237977 | -0.231557 | -0.248803 | 1.807298 | -0.689071 | -0.276255 | -0.235753 | -0.865341 | 1.072824 | -0.799449 | -0.347317 | -0.276255 |
2 | -1.328610 | 0.430219 | -1.505660 | -0.451666 | -0.303916 | -0.972108 | -0.237907 | -0.231557 | -0.249115 | 1.782005 | -0.668144 | -0.214881 | -0.235182 | -0.865341 | 0.342583 | -0.792795 | -0.347317 | -0.214881 |
3 | -1.163304 | 0.430219 | -1.381998 | -0.451666 | -0.303916 | -0.972108 | -0.238047 | -0.231557 | -0.236297 | 1.857216 | -0.634136 | -0.302558 | -0.234848 | -0.865341 | 0.535429 | -0.773271 | -0.347317 | -0.302558 |
4 | -0.997998 | 0.430219 | -1.281522 | -0.451666 | -0.303916 | -0.972108 | -0.237977 | -0.231557 | -0.234030 | 1.844149 | -0.612630 | -0.255797 | -0.234654 | -0.865341 | 0.532713 | -0.748810 | -0.347317 | -0.255797 |
df_all_cat_enc = pd.get_dummies(df_all_cat)
df_all_cat_enc.head()
entity_Afghanistan | entity_Albania | entity_Antigua and Barbuda | entity_Argentina | entity_Armenia | entity_Aruba | entity_Austria | entity_Azerbaijan | entity_Bahamas | entity_Bahrain | entity_Bangladesh | entity_Barbados | entity_Belarus | entity_Belgium | entity_Belize | entity_Bermuda | entity_Bhutan | entity_Bosnia and Herzegovina | entity_Brazil | entity_Bulgaria | entity_Cambodia | entity_Canada | entity_Cayman Islands | entity_Chile | entity_China | entity_Colombia | entity_Costa Rica | entity_Croatia | entity_Cuba | entity_Cyprus | entity_Czechia | entity_Denmark | entity_Dominica | entity_Dominican Republic | entity_Ecuador | entity_El Salvador | entity_Estonia | entity_Finland | entity_France | entity_French Guiana | entity_Georgia | entity_Germany | entity_Greece | entity_Grenada | entity_Guatemala | entity_Guyana | entity_Haiti | entity_Honduras | entity_Hungary | entity_Iceland | entity_India | entity_Indonesia | entity_Iraq | entity_Ireland | entity_Israel | entity_Italy | entity_Jamaica | entity_Japan | entity_Jordan | entity_Kazakhstan | entity_Kuwait | entity_Kyrgyzstan | entity_Latvia | entity_Lebanon | entity_Lithuania | entity_Luxembourg | entity_Malaysia | entity_Maldives | entity_Malta | entity_Mexico | entity_Mongolia | entity_Montenegro | entity_Myanmar | entity_Nepal | entity_Netherlands | entity_Nicaragua | entity_North Macedonia | entity_Norway | entity_Oman | entity_Pakistan | entity_Panama | entity_Paraguay | entity_Peru | entity_Philippines | entity_Poland | entity_Portugal | entity_Puerto Rico | entity_Qatar | entity_Romania | entity_Saint Kitts and Nevis | entity_Saint Lucia | entity_Saint Vincent and the Grenadines | entity_Saudi Arabia | entity_Serbia | entity_Singapore | entity_Slovakia | entity_Slovenia | entity_Spain | entity_Sri Lanka | entity_Suriname | entity_Sweden | entity_Switzerland | entity_Tajikistan | entity_Thailand | entity_Trinidad and Tobago | entity_Turkey | entity_Turkmenistan | entity_Ukraine | entity_United Arab Emirates | entity_United Kingdom | entity_United States | entity_Uruguay | entity_Uzbekistan | entity_Yemen | continent_Asia | continent_Europe | continent_North America | continent_South America | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
1 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
2 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
3 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
4 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
df_all_num_normalized.head()
year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | landarea(km2) | primary_energy_intensity | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | -1.659221 | 0.430219 | -1.698883 | -0.451666 | -0.303916 | -0.972108 | -0.237954 | -0.231557 | -0.240752 | 1.824784 | -0.678869 | -0.206113 | -0.235817 | -0.865341 | 0.810977 | -0.806636 | -0.347317 | -0.206113 |
1 | -1.493915 | 0.430219 | -1.610001 | -0.451666 | -0.303916 | -0.972108 | -0.237977 | -0.231557 | -0.248803 | 1.807298 | -0.689071 | -0.276255 | -0.235753 | -0.865341 | 1.072824 | -0.799449 | -0.347317 | -0.276255 |
2 | -1.328610 | 0.430219 | -1.505660 | -0.451666 | -0.303916 | -0.972108 | -0.237907 | -0.231557 | -0.249115 | 1.782005 | -0.668144 | -0.214881 | -0.235182 | -0.865341 | 0.342583 | -0.792795 | -0.347317 | -0.214881 |
3 | -1.163304 | 0.430219 | -1.381998 | -0.451666 | -0.303916 | -0.972108 | -0.238047 | -0.231557 | -0.236297 | 1.857216 | -0.634136 | -0.302558 | -0.234848 | -0.865341 | 0.535429 | -0.773271 | -0.347317 | -0.302558 |
4 | -0.997998 | 0.430219 | -1.281522 | -0.451666 | -0.303916 | -0.972108 | -0.237977 | -0.231557 | -0.234030 | 1.844149 | -0.612630 | -0.255797 | -0.234654 | -0.865341 | 0.532713 | -0.748810 | -0.347317 | -0.255797 |
X=np.concatenate([df_all_num_normalized, df_all_cat_enc],axis=1)
X
array([[-1.65922091, 0.43021915, -1.69888305, ..., 1. , 0. , 0. ], [-1.49391521, 0.43021915, -1.61000064, ..., 1. , 0. , 0. ], [-1.32860951, 0.43021915, -1.50566041, ..., 1. , 0. , 0. ], ..., [ 1.31628163, 0.41561275, 0.68934879, ..., 0. , 0. , 1. ], [ 1.48158733, 0.42291595, 0.68934879, ..., 0. , 0. , 1. ], [ 1.64689303, 0.43021915, 0.68934879, ..., 0. , 0. , 1. ]])
len(X[0])
136
test= pd.concat([df_all_num_normalized, df_all_cat_enc],axis=1)
test.head()
year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | landarea(km2) | primary_energy_intensity | entity_Afghanistan | entity_Albania | entity_Antigua and Barbuda | entity_Argentina | entity_Armenia | entity_Aruba | entity_Austria | entity_Azerbaijan | entity_Bahamas | entity_Bahrain | entity_Bangladesh | entity_Barbados | entity_Belarus | entity_Belgium | entity_Belize | entity_Bermuda | entity_Bhutan | entity_Bosnia and Herzegovina | entity_Brazil | entity_Bulgaria | entity_Cambodia | entity_Canada | entity_Cayman Islands | entity_Chile | entity_China | entity_Colombia | entity_Costa Rica | entity_Croatia | entity_Cuba | entity_Cyprus | entity_Czechia | entity_Denmark | entity_Dominica | entity_Dominican Republic | entity_Ecuador | entity_El Salvador | entity_Estonia | entity_Finland | entity_France | entity_French Guiana | entity_Georgia | entity_Germany | entity_Greece | entity_Grenada | entity_Guatemala | entity_Guyana | entity_Haiti | entity_Honduras | entity_Hungary | entity_Iceland | entity_India | entity_Indonesia | entity_Iraq | entity_Ireland | entity_Israel | entity_Italy | entity_Jamaica | entity_Japan | entity_Jordan | entity_Kazakhstan | entity_Kuwait | entity_Kyrgyzstan | entity_Latvia | entity_Lebanon | entity_Lithuania | entity_Luxembourg | entity_Malaysia | entity_Maldives | entity_Malta | entity_Mexico | entity_Mongolia | entity_Montenegro | entity_Myanmar | entity_Nepal | entity_Netherlands | entity_Nicaragua | entity_North Macedonia | entity_Norway | entity_Oman | entity_Pakistan | entity_Panama | entity_Paraguay | entity_Peru | entity_Philippines | entity_Poland | entity_Portugal | entity_Puerto Rico | entity_Qatar | entity_Romania | entity_Saint Kitts and Nevis | entity_Saint Lucia | entity_Saint Vincent and the Grenadines | entity_Saudi Arabia | entity_Serbia | entity_Singapore | entity_Slovakia | entity_Slovenia | entity_Spain | entity_Sri Lanka | entity_Suriname | entity_Sweden | entity_Switzerland | entity_Tajikistan | entity_Thailand | entity_Trinidad and Tobago | entity_Turkey | entity_Turkmenistan | entity_Ukraine | entity_United Arab Emirates | entity_United Kingdom | entity_United States | entity_Uruguay | entity_Uzbekistan | entity_Yemen | continent_Asia | continent_Europe | continent_North America | continent_South America | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | -1.659221 | 0.430219 | -1.698883 | -0.451666 | -0.303916 | -0.972108 | -0.237954 | -0.231557 | -0.240752 | 1.824784 | -0.678869 | -0.206113 | -0.235817 | -0.865341 | 0.810977 | -0.806636 | -0.347317 | -0.206113 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
1 | -1.493915 | 0.430219 | -1.610001 | -0.451666 | -0.303916 | -0.972108 | -0.237977 | -0.231557 | -0.248803 | 1.807298 | -0.689071 | -0.276255 | -0.235753 | -0.865341 | 1.072824 | -0.799449 | -0.347317 | -0.276255 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
2 | -1.328610 | 0.430219 | -1.505660 | -0.451666 | -0.303916 | -0.972108 | -0.237907 | -0.231557 | -0.249115 | 1.782005 | -0.668144 | -0.214881 | -0.235182 | -0.865341 | 0.342583 | -0.792795 | -0.347317 | -0.214881 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
3 | -1.163304 | 0.430219 | -1.381998 | -0.451666 | -0.303916 | -0.972108 | -0.238047 | -0.231557 | -0.236297 | 1.857216 | -0.634136 | -0.302558 | -0.234848 | -0.865341 | 0.535429 | -0.773271 | -0.347317 | -0.302558 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
4 | -0.997998 | 0.430219 | -1.281522 | -0.451666 | -0.303916 | -0.972108 | -0.237977 | -0.231557 | -0.234030 | 1.844149 | -0.612630 | -0.255797 | -0.234654 | -0.865341 | 0.532713 | -0.748810 | -0.347317 | -0.255797 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 |
K = range(1,11)
inertia = []
for k in K:
kmeans = KMeans(n_clusters = k,
random_state = 42)
kmeans.fit(X)
inertia.append(kmeans.inertia_)
inertia
[46493.100000000006, 39060.869614359995, 34166.930037454214, 30820.638132463202, 28297.359058094877, 26554.979696368428, 24991.745665644794, 23868.138746913566, 22306.87750205997, 21344.27270385504]
plt.figure(figsize=(16,8))
plt.plot(K,inertia,'bx-')
plt.xlabel('k')
plt.ylabel('inertia')
plt.xticks(np.arange(min(K),max(K)+1, 1.0))
plt.title('Elbow Method showing the optimal k')
Text(0.5, 1.0, 'Elbow Method showing the optimal k')
kmeans = KMeans(n_clusters=4, random_state=1234, verbose=1)
kmeans.fit(X)
Initialization complete Iteration 0, inertia 47091.542866271775 Iteration 1, inertia 37796.75104816156 Iteration 2, inertia 37194.10982729161 Iteration 3, inertia 36611.368944543785 Iteration 4, inertia 36318.467927294725 Iteration 5, inertia 36173.89036128343 Iteration 6, inertia 36088.548141610925 Iteration 7, inertia 36063.90541245332 Iteration 8, inertia 36051.20366330692 Iteration 9, inertia 36045.472266248966 Iteration 10, inertia 36044.04397231238 Iteration 11, inertia 36043.62762429331 Iteration 12, inertia 36043.285364095034 Iteration 13, inertia 36043.01843756064 Iteration 14, inertia 36042.99390909351 Iteration 15, inertia 36042.97472111219 Iteration 16, inertia 36042.936855300504 Iteration 17, inertia 36042.80660461281 Iteration 18, inertia 36042.70105875199 Iteration 19, inertia 36042.67094459025 Iteration 20, inertia 36042.645758735365 Iteration 21, inertia 36042.63202098105 Converged at iteration 21: strict convergence. Initialization complete Iteration 0, inertia 42811.94650909942 Iteration 1, inertia 33063.00420679177 Iteration 2, inertia 32854.158059853544 Iteration 3, inertia 32589.784871011158 Iteration 4, inertia 31960.55288914963 Iteration 5, inertia 31539.031080309247 Iteration 6, inertia 31331.299474540163 Iteration 7, inertia 31264.521842183873 Iteration 8, inertia 31245.84919146684 Iteration 9, inertia 31238.897464387537 Iteration 10, inertia 31237.278945578895 Iteration 11, inertia 31237.022960439437 Iteration 12, inertia 31236.981928739333 Iteration 13, inertia 31236.864171765304 Iteration 14, inertia 31236.813860988394 Converged at iteration 14: strict convergence. Initialization complete Iteration 0, inertia 44335.637340646375 Iteration 1, inertia 31607.268619785966 Iteration 2, inertia 31382.62735297526 Iteration 3, inertia 31281.411540396493 Iteration 4, inertia 31247.485123587623 Iteration 5, inertia 31235.21288366886 Iteration 6, inertia 31230.461623355397 Iteration 7, inertia 31228.400156780022 Iteration 8, inertia 31228.150327721338 Iteration 9, inertia 31227.808226067682 Iteration 10, inertia 31227.6979285061 Iteration 11, inertia 31227.674438175032 Iteration 12, inertia 31227.65266782903 Converged at iteration 12: center shift 1.3180553947320061e-05 within tolerance 1.4485636839481595e-05. Initialization complete Iteration 0, inertia 46109.4343476128 Iteration 1, inertia 31563.924975787253 Iteration 2, inertia 31323.781205297644 Iteration 3, inertia 31268.64358897337 Iteration 4, inertia 31250.806355693265 Iteration 5, inertia 31241.5265733274 Iteration 6, inertia 31235.478639362373 Iteration 7, inertia 31232.94783305141 Iteration 8, inertia 31230.046725430366 Iteration 9, inertia 31229.50908448498 Iteration 10, inertia 31229.078834844517 Iteration 11, inertia 31228.722408611608 Iteration 12, inertia 31228.60828395903 Iteration 13, inertia 31228.56858437405 Iteration 14, inertia 31228.463881401705 Iteration 15, inertia 31228.349149160596 Iteration 16, inertia 31228.23884904843 Iteration 17, inertia 31228.102926130283 Iteration 18, inertia 31227.927746893434 Iteration 19, inertia 31227.828349298594 Iteration 20, inertia 31227.80991842606 Converged at iteration 20: strict convergence. Initialization complete Iteration 0, inertia 47873.14349592235 Iteration 1, inertia 38177.478990277 Iteration 2, inertia 37801.117222046116 Iteration 3, inertia 37552.89758520856 Iteration 4, inertia 37292.916533685835 Iteration 5, inertia 37102.49820135308 Iteration 6, inertia 37071.60256343434 Iteration 7, inertia 37040.12459196425 Iteration 8, inertia 36992.57175471328 Iteration 9, inertia 36953.294132284514 Iteration 10, inertia 36928.62937172566 Iteration 11, inertia 36919.825822270956 Iteration 12, inertia 36916.78705821484 Iteration 13, inertia 36912.36032939525 Iteration 14, inertia 36909.26790222391 Iteration 15, inertia 36899.56456924505 Iteration 16, inertia 36821.356065477834 Iteration 17, inertia 36784.13780191843 Iteration 18, inertia 36746.16039555599 Iteration 19, inertia 36688.887479205325 Iteration 20, inertia 36613.07558552302 Iteration 21, inertia 36477.670984793054 Iteration 22, inertia 36228.835420852825 Iteration 23, inertia 35985.42169891661 Iteration 24, inertia 35633.35188988839 Iteration 25, inertia 35208.629786341524 Iteration 26, inertia 35071.401777115876 Iteration 27, inertia 34910.198552314665 Iteration 28, inertia 34751.19154217254 Iteration 29, inertia 34566.0112793953 Iteration 30, inertia 34258.75798697361 Iteration 31, inertia 33736.9687012815 Iteration 32, inertia 33517.34736086923 Iteration 33, inertia 32631.749256207622 Iteration 34, inertia 31903.69706375056 Iteration 35, inertia 31257.681539767887 Iteration 36, inertia 31239.87636698039 Iteration 37, inertia 31237.51910781564 Iteration 38, inertia 31237.108618536025 Iteration 39, inertia 31236.74948893649 Iteration 40, inertia 31236.72441460044 Converged at iteration 40: strict convergence. Initialization complete Iteration 0, inertia 51847.39232555053 Iteration 1, inertia 33149.63159807736 Iteration 2, inertia 32753.044700628954 Iteration 3, inertia 32652.848148705576 Iteration 4, inertia 32571.818721208012 Iteration 5, inertia 32395.47190737905 Iteration 6, inertia 32133.638691278666 Iteration 7, inertia 31700.489334879254 Iteration 8, inertia 31383.61199674766 Iteration 9, inertia 31282.820203788466 Iteration 10, inertia 31204.84226233569 Iteration 11, inertia 31074.92274567734 Iteration 12, inertia 30923.923283108074 Iteration 13, inertia 30907.906325943266 Iteration 14, inertia 30896.58924025233 Iteration 15, inertia 30882.32210330447 Iteration 16, inertia 30855.244859915147 Iteration 17, inertia 30838.566102822475 Iteration 18, inertia 30833.85481176126 Iteration 19, inertia 30832.3553202651 Iteration 20, inertia 30832.191692006076 Iteration 21, inertia 30832.117775037434 Converged at iteration 21: strict convergence. Initialization complete Iteration 0, inertia 47033.727617865574 Iteration 1, inertia 37488.74491264901 Iteration 2, inertia 37036.45141227646 Iteration 3, inertia 36746.37843149467 Iteration 4, inertia 36642.90533868252 Iteration 5, inertia 36590.8495019705 Iteration 6, inertia 36555.31609207527 Iteration 7, inertia 36521.66378420297 Iteration 8, inertia 36465.63702475608 Iteration 9, inertia 36396.34086542008 Iteration 10, inertia 36277.18891317554 Iteration 11, inertia 35813.08898063973 Iteration 12, inertia 34958.34834923362 Iteration 13, inertia 34462.29928955963 Iteration 14, inertia 34007.47370128607 Iteration 15, inertia 33641.2475258636 Iteration 16, inertia 33125.44322466759 Iteration 17, inertia 32095.56505075077 Iteration 18, inertia 31731.690537755432 Iteration 19, inertia 31252.97178812277 Iteration 20, inertia 31231.606533364644 Iteration 21, inertia 31228.82692895011 Iteration 22, inertia 31228.51459593262 Iteration 23, inertia 31228.29857147696 Iteration 24, inertia 31228.190685092573 Iteration 25, inertia 31227.998684074482 Iteration 26, inertia 31227.859956141867 Iteration 27, inertia 31227.764593505366 Iteration 28, inertia 31227.685238823775 Iteration 29, inertia 31227.66441547741 Iteration 30, inertia 31227.64696478688 Converged at iteration 30: center shift 1.3212186310730134e-05 within tolerance 1.4485636839481595e-05. Initialization complete Iteration 0, inertia 47585.84174624087 Iteration 1, inertia 32813.26757339154 Iteration 2, inertia 32294.02428263995 Iteration 3, inertia 31964.334796487372 Iteration 4, inertia 31267.566163160685 Iteration 5, inertia 31244.289004974173 Iteration 6, inertia 31239.308779938612 Iteration 7, inertia 31238.74478020834 Iteration 8, inertia 31236.966312728196 Iteration 9, inertia 31235.962606985362 Iteration 10, inertia 31235.32326461521 Iteration 11, inertia 31235.09318253754 Iteration 12, inertia 31235.03523380363 Converged at iteration 12: strict convergence. Initialization complete Iteration 0, inertia 42821.58470158292 Iteration 1, inertia 32344.570366023232 Iteration 2, inertia 31554.540987098706 Iteration 3, inertia 31352.777771493165 Iteration 4, inertia 31265.868452038987 Iteration 5, inertia 31241.796699247505 Iteration 6, inertia 31233.603204197403 Iteration 7, inertia 31229.774937553662 Iteration 8, inertia 31228.533681724344 Iteration 9, inertia 31228.118135699504 Iteration 10, inertia 31227.791182170913 Iteration 11, inertia 31227.682822420888 Iteration 12, inertia 31227.66441547741 Iteration 13, inertia 31227.64696478688 Converged at iteration 13: center shift 1.3212186310728894e-05 within tolerance 1.4485636839481595e-05. Initialization complete Iteration 0, inertia 47973.82650769352 Iteration 1, inertia 37224.62435259784 Iteration 2, inertia 36495.7283422889 Iteration 3, inertia 36156.38637668906 Iteration 4, inertia 35881.92734629006 Iteration 5, inertia 35694.30260776198 Iteration 6, inertia 35575.75415916623 Iteration 7, inertia 35535.946019304654 Iteration 8, inertia 35521.88867285803 Iteration 9, inertia 35516.65654530879 Iteration 10, inertia 35513.39019679838 Iteration 11, inertia 35511.98522868887 Iteration 12, inertia 35511.717348321996 Iteration 13, inertia 35511.05141292716 Iteration 14, inertia 35510.577399530564 Iteration 15, inertia 35510.356740497475 Iteration 16, inertia 35510.24826006764 Converged at iteration 16: strict convergence.
KMeans(n_clusters=4, random_state=1234, verbose=1)
kmeans.inertia_
30832.117775037434
#Making predictions
clusters = kmeans.predict(X)
# Check the size of the clusters
pd.Series(clusters).value_counts().sort_index()
0 1643 1 39 2 448 3 230 dtype: int64
X_data= pd.DataFrame(X)
X_data.head()
0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | 72 | 73 | 74 | 75 | 76 | 77 | 78 | 79 | 80 | 81 | 82 | 83 | 84 | 85 | 86 | 87 | 88 | 89 | 90 | 91 | 92 | 93 | 94 | 95 | 96 | 97 | 98 | 99 | 100 | 101 | 102 | 103 | 104 | 105 | 106 | 107 | 108 | 109 | 110 | 111 | 112 | 113 | 114 | 115 | 116 | 117 | 118 | 119 | 120 | 121 | 122 | 123 | 124 | 125 | 126 | 127 | 128 | 129 | 130 | 131 | 132 | 133 | 134 | 135 | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | -1.659221 | 0.430219 | -1.698883 | -0.451666 | -0.303916 | -0.972108 | -0.237954 | -0.231557 | -0.240752 | 1.824784 | -0.678869 | -0.206113 | -0.235817 | -0.865341 | 0.810977 | -0.806636 | -0.347317 | -0.206113 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 |
1 | -1.493915 | 0.430219 | -1.610001 | -0.451666 | -0.303916 | -0.972108 | -0.237977 | -0.231557 | -0.248803 | 1.807298 | -0.689071 | -0.276255 | -0.235753 | -0.865341 | 1.072824 | -0.799449 | -0.347317 | -0.276255 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 |
2 | -1.328610 | 0.430219 | -1.505660 | -0.451666 | -0.303916 | -0.972108 | -0.237907 | -0.231557 | -0.249115 | 1.782005 | -0.668144 | -0.214881 | -0.235182 | -0.865341 | 0.342583 | -0.792795 | -0.347317 | -0.214881 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 |
3 | -1.163304 | 0.430219 | -1.381998 | -0.451666 | -0.303916 | -0.972108 | -0.238047 | -0.231557 | -0.236297 | 1.857216 | -0.634136 | -0.302558 | -0.234848 | -0.865341 | 0.535429 | -0.773271 | -0.347317 | -0.302558 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 |
4 | -0.997998 | 0.430219 | -1.281522 | -0.451666 | -0.303916 | -0.972108 | -0.237977 | -0.231557 | -0.234030 | 1.844149 | -0.612630 | -0.255797 | -0.234654 | -0.865341 | 0.532713 | -0.748810 | -0.347317 | -0.255797 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 |
X_df = pd.DataFrame(X)
X_df["cluster"] = clusters
X_df.head()
0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | 72 | 73 | 74 | 75 | 76 | 77 | 78 | 79 | 80 | 81 | 82 | 83 | 84 | 85 | 86 | 87 | 88 | 89 | 90 | 91 | 92 | 93 | 94 | 95 | 96 | 97 | 98 | 99 | 100 | 101 | 102 | 103 | 104 | 105 | 106 | 107 | 108 | 109 | 110 | 111 | 112 | 113 | 114 | 115 | 116 | 117 | 118 | 119 | 120 | 121 | 122 | 123 | 124 | 125 | 126 | 127 | 128 | 129 | 130 | 131 | 132 | 133 | 134 | 135 | cluster | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | -1.659221 | 0.430219 | -1.698883 | -0.451666 | -0.303916 | -0.972108 | -0.237954 | -0.231557 | -0.240752 | 1.824784 | -0.678869 | -0.206113 | -0.235817 | -0.865341 | 0.810977 | -0.806636 | -0.347317 | -0.206113 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0 |
1 | -1.493915 | 0.430219 | -1.610001 | -0.451666 | -0.303916 | -0.972108 | -0.237977 | -0.231557 | -0.248803 | 1.807298 | -0.689071 | -0.276255 | -0.235753 | -0.865341 | 1.072824 | -0.799449 | -0.347317 | -0.276255 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0 |
2 | -1.328610 | 0.430219 | -1.505660 | -0.451666 | -0.303916 | -0.972108 | -0.237907 | -0.231557 | -0.249115 | 1.782005 | -0.668144 | -0.214881 | -0.235182 | -0.865341 | 0.342583 | -0.792795 | -0.347317 | -0.214881 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0 |
3 | -1.163304 | 0.430219 | -1.381998 | -0.451666 | -0.303916 | -0.972108 | -0.238047 | -0.231557 | -0.236297 | 1.857216 | -0.634136 | -0.302558 | -0.234848 | -0.865341 | 0.535429 | -0.773271 | -0.347317 | -0.302558 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0 |
4 | -0.997998 | 0.430219 | -1.281522 | -0.451666 | -0.303916 | -0.972108 | -0.237977 | -0.231557 | -0.234030 | 1.844149 | -0.612630 | -0.255797 | -0.234654 | -0.865341 | 0.532713 | -0.748810 | -0.347317 | -0.255797 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0 |
X_df[X_df['cluster']==0].head()
0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | 72 | 73 | 74 | 75 | 76 | 77 | 78 | 79 | 80 | 81 | 82 | 83 | 84 | 85 | 86 | 87 | 88 | 89 | 90 | 91 | 92 | 93 | 94 | 95 | 96 | 97 | 98 | 99 | 100 | 101 | 102 | 103 | 104 | 105 | 106 | 107 | 108 | 109 | 110 | 111 | 112 | 113 | 114 | 115 | 116 | 117 | 118 | 119 | 120 | 121 | 122 | 123 | 124 | 125 | 126 | 127 | 128 | 129 | 130 | 131 | 132 | 133 | 134 | 135 | cluster | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | -1.659221 | 0.430219 | -1.698883 | -0.451666 | -0.303916 | -0.972108 | -0.237954 | -0.231557 | -0.240752 | 1.824784 | -0.678869 | -0.206113 | -0.235817 | -0.865341 | 0.810977 | -0.806636 | -0.347317 | -0.206113 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0 |
1 | -1.493915 | 0.430219 | -1.610001 | -0.451666 | -0.303916 | -0.972108 | -0.237977 | -0.231557 | -0.248803 | 1.807298 | -0.689071 | -0.276255 | -0.235753 | -0.865341 | 1.072824 | -0.799449 | -0.347317 | -0.276255 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0 |
2 | -1.328610 | 0.430219 | -1.505660 | -0.451666 | -0.303916 | -0.972108 | -0.237907 | -0.231557 | -0.249115 | 1.782005 | -0.668144 | -0.214881 | -0.235182 | -0.865341 | 0.342583 | -0.792795 | -0.347317 | -0.214881 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0 |
3 | -1.163304 | 0.430219 | -1.381998 | -0.451666 | -0.303916 | -0.972108 | -0.238047 | -0.231557 | -0.236297 | 1.857216 | -0.634136 | -0.302558 | -0.234848 | -0.865341 | 0.535429 | -0.773271 | -0.347317 | -0.302558 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0 |
4 | -0.997998 | 0.430219 | -1.281522 | -0.451666 | -0.303916 | -0.972108 | -0.237977 | -0.231557 | -0.234030 | 1.844149 | -0.612630 | -0.255797 | -0.234654 | -0.865341 | 0.532713 | -0.748810 | -0.347317 | -0.255797 | 0.0 | 1.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 1.0 | 0.0 | 0.0 | 0 |
K = range(1,11)
inertia = []
for k in K:
kmeans = KMeans(n_clusters = k,
random_state = 42)
kmeans.fit(test)
inertia.append(kmeans.inertia_)
inertia
[46493.100000000006, 39060.869614359995, 34166.930037454214, 30820.638132463202, 28297.359058094877, 26554.979696368428, 24991.745665644794, 23868.138746913566, 22306.87750205997, 21344.27270385504]
clusters_test = kmeans.predict(test)
pd.Series(clusters_test).value_counts().sort_index()
0 289 1 310 2 37 3 133 4 75 5 694 6 59 7 47 8 140 9 576 dtype: int64
test2=test.copy()
test2['clusters_test']=clusters_test
pd.set_option('display.max_rows', None) # Mostrar todas las filas
pd.set_option('display.max_columns', None) # Mostrar todas las columnas
selected_rows = test2[test2['clusters_test'] == 9]
selected_rows_reset = selected_rows.reset_index(drop=True)
selected_rows_reset.head()
year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | landarea(km2) | primary_energy_intensity | entity_Afghanistan | entity_Albania | entity_Antigua and Barbuda | entity_Argentina | entity_Armenia | entity_Aruba | entity_Austria | entity_Azerbaijan | entity_Bahamas | entity_Bahrain | entity_Bangladesh | entity_Barbados | entity_Belarus | entity_Belgium | entity_Belize | entity_Bermuda | entity_Bhutan | entity_Bosnia and Herzegovina | entity_Brazil | entity_Bulgaria | entity_Cambodia | entity_Canada | entity_Cayman Islands | entity_Chile | entity_China | entity_Colombia | entity_Costa Rica | entity_Croatia | entity_Cuba | entity_Cyprus | entity_Czechia | entity_Denmark | entity_Dominica | entity_Dominican Republic | entity_Ecuador | entity_El Salvador | entity_Estonia | entity_Finland | entity_France | entity_French Guiana | entity_Georgia | entity_Germany | entity_Greece | entity_Grenada | entity_Guatemala | entity_Guyana | entity_Haiti | entity_Honduras | entity_Hungary | entity_Iceland | entity_India | entity_Indonesia | entity_Iraq | entity_Ireland | entity_Israel | entity_Italy | entity_Jamaica | entity_Japan | entity_Jordan | entity_Kazakhstan | entity_Kuwait | entity_Kyrgyzstan | entity_Latvia | entity_Lebanon | entity_Lithuania | entity_Luxembourg | entity_Malaysia | entity_Maldives | entity_Malta | entity_Mexico | entity_Mongolia | entity_Montenegro | entity_Myanmar | entity_Nepal | entity_Netherlands | entity_Nicaragua | entity_North Macedonia | entity_Norway | entity_Oman | entity_Pakistan | entity_Panama | entity_Paraguay | entity_Peru | entity_Philippines | entity_Poland | entity_Portugal | entity_Puerto Rico | entity_Qatar | entity_Romania | entity_Saint Kitts and Nevis | entity_Saint Lucia | entity_Saint Vincent and the Grenadines | entity_Saudi Arabia | entity_Serbia | entity_Singapore | entity_Slovakia | entity_Slovenia | entity_Spain | entity_Sri Lanka | entity_Suriname | entity_Sweden | entity_Switzerland | entity_Tajikistan | entity_Thailand | entity_Trinidad and Tobago | entity_Turkey | entity_Turkmenistan | entity_Ukraine | entity_United Arab Emirates | entity_United Kingdom | entity_United States | entity_Uruguay | entity_Uzbekistan | entity_Yemen | continent_Asia | continent_Europe | continent_North America | continent_South America | clusters_test | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | -1.659221 | 0.430219 | -1.698883 | -0.451666 | -0.303916 | -0.972108 | -0.237954 | -0.231557 | -0.240752 | 1.824784 | -0.678869 | -0.206113 | -0.235817 | -0.865341 | 0.810977 | -0.806636 | -0.347317 | -0.206113 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 9 |
1 | -1.493915 | 0.430219 | -1.610001 | -0.451666 | -0.303916 | -0.972108 | -0.237977 | -0.231557 | -0.248803 | 1.807298 | -0.689071 | -0.276255 | -0.235753 | -0.865341 | 1.072824 | -0.799449 | -0.347317 | -0.276255 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 9 |
2 | -1.328610 | 0.430219 | -1.505660 | -0.451666 | -0.303916 | -0.972108 | -0.237907 | -0.231557 | -0.249115 | 1.782005 | -0.668144 | -0.214881 | -0.235182 | -0.865341 | 0.342583 | -0.792795 | -0.347317 | -0.214881 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 9 |
3 | -1.163304 | 0.430219 | -1.381998 | -0.451666 | -0.303916 | -0.972108 | -0.238047 | -0.231557 | -0.236297 | 1.857216 | -0.634136 | -0.302558 | -0.234848 | -0.865341 | 0.535429 | -0.773271 | -0.347317 | -0.302558 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 9 |
4 | -0.997998 | 0.430219 | -1.281522 | -0.451666 | -0.303916 | -0.972108 | -0.237977 | -0.231557 | -0.234030 | 1.844149 | -0.612630 | -0.255797 | -0.234654 | -0.865341 | 0.532713 | -0.748810 | -0.347317 | -0.255797 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 9 |
test2[test2['clusters_test']==1].head()
year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | landarea(km2) | primary_energy_intensity | entity_Afghanistan | entity_Albania | entity_Antigua and Barbuda | entity_Argentina | entity_Armenia | entity_Aruba | entity_Austria | entity_Azerbaijan | entity_Bahamas | entity_Bahrain | entity_Bangladesh | entity_Barbados | entity_Belarus | entity_Belgium | entity_Belize | entity_Bermuda | entity_Bhutan | entity_Bosnia and Herzegovina | entity_Brazil | entity_Bulgaria | entity_Cambodia | entity_Canada | entity_Cayman Islands | entity_Chile | entity_China | entity_Colombia | entity_Costa Rica | entity_Croatia | entity_Cuba | entity_Cyprus | entity_Czechia | entity_Denmark | entity_Dominica | entity_Dominican Republic | entity_Ecuador | entity_El Salvador | entity_Estonia | entity_Finland | entity_France | entity_French Guiana | entity_Georgia | entity_Germany | entity_Greece | entity_Grenada | entity_Guatemala | entity_Guyana | entity_Haiti | entity_Honduras | entity_Hungary | entity_Iceland | entity_India | entity_Indonesia | entity_Iraq | entity_Ireland | entity_Israel | entity_Italy | entity_Jamaica | entity_Japan | entity_Jordan | entity_Kazakhstan | entity_Kuwait | entity_Kyrgyzstan | entity_Latvia | entity_Lebanon | entity_Lithuania | entity_Luxembourg | entity_Malaysia | entity_Maldives | entity_Malta | entity_Mexico | entity_Mongolia | entity_Montenegro | entity_Myanmar | entity_Nepal | entity_Netherlands | entity_Nicaragua | entity_North Macedonia | entity_Norway | entity_Oman | entity_Pakistan | entity_Panama | entity_Paraguay | entity_Peru | entity_Philippines | entity_Poland | entity_Portugal | entity_Puerto Rico | entity_Qatar | entity_Romania | entity_Saint Kitts and Nevis | entity_Saint Lucia | entity_Saint Vincent and the Grenadines | entity_Saudi Arabia | entity_Serbia | entity_Singapore | entity_Slovakia | entity_Slovenia | entity_Spain | entity_Sri Lanka | entity_Suriname | entity_Sweden | entity_Switzerland | entity_Tajikistan | entity_Thailand | entity_Trinidad and Tobago | entity_Turkey | entity_Turkmenistan | entity_Ukraine | entity_United Arab Emirates | entity_United Kingdom | entity_United States | entity_Uruguay | entity_Uzbekistan | entity_Yemen | continent_Asia | continent_Europe | continent_North America | continent_South America | clusters_test | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
763 | -1.659221 | -6.755135 | -2.935508 | -0.416814 | -0.303805 | 1.102729 | -0.237907 | -0.231557 | -0.273893 | 0.882865 | -0.904892 | -0.933833 | -0.238412 | -0.865341 | -0.539223 | -0.858887 | -0.000622 | -0.933833 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 1 |
764 | -1.493915 | -6.575405 | -2.896863 | -0.418175 | -0.303197 | 1.130860 | -0.238071 | -0.231557 | -0.272408 | 1.452682 | -0.906594 | -0.904607 | -0.238445 | -0.865341 | -0.539223 | -0.858887 | -0.000622 | -0.904607 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 1 |
765 | -1.328610 | -6.185809 | -2.858219 | -0.419649 | -0.282084 | 0.772526 | -0.237977 | -0.231557 | -0.271939 | 1.343914 | -0.907268 | -1.003975 | -0.238122 | -0.865341 | -0.539223 | -0.850565 | -0.000622 | -1.003975 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 1 |
766 | -1.163304 | -5.796597 | -2.807981 | -0.421086 | -0.160378 | 0.718568 | -0.237556 | -0.231557 | -0.271392 | 0.915129 | -0.906773 | -1.003975 | -0.237917 | -0.865341 | 1.177588 | -0.850043 | -0.000622 | -1.003975 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 1 |
767 | -0.997998 | -5.407594 | -2.753879 | -0.422371 | -0.160378 | 1.068140 | -0.237509 | -0.231557 | -0.271939 | 0.790785 | -0.907440 | -1.062426 | -0.238122 | -0.865341 | -0.264348 | -0.849084 | -0.000622 | -1.062426 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 1 |
df_all.shape
(2360, 20)
df_all_num.head()
year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | landarea(km2) | primary_energy_intensity | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | 2000 | 100.0 | 38.2 | 0.0 | 0.0 | 0.0 | 0.14 | 0.0 | 4.55 | 97.014930 | 9029.4375 | 4.13 | 3170.000000 | 0.0 | 6.946217 | 1126.683340 | 28748.0 | 4.13 |
1 | 2001 | 100.0 | 40.5 | 0.0 | 0.0 | 0.0 | 0.13 | 0.0 | 3.52 | 96.438354 | 8635.5320 | 3.89 | 3230.000000 | 0.0 | 8.293313 | 1281.659826 | 28748.0 | 3.89 |
2 | 2002 | 100.0 | 43.2 | 0.0 | 0.0 | 0.0 | 0.16 | 0.0 | 3.48 | 95.604390 | 9443.5550 | 4.10 | 3759.999990 | 0.0 | 4.536524 | 1425.124219 | 28748.0 | 4.10 |
3 | 2003 | 100.0 | 46.4 | 0.0 | 0.0 | 0.0 | 0.10 | 0.0 | 5.12 | 98.084300 | 10756.6120 | 3.80 | 4070.000172 | 0.0 | 5.528637 | 1846.120121 | 28748.0 | 3.80 |
4 | 2004 | 100.0 | 49.0 | 0.0 | 0.0 | 0.0 | 0.13 | 0.0 | 5.41 | 97.653430 | 11586.9510 | 3.96 | 4250.000000 | 0.0 | 5.514668 | 2373.581292 | 28748.0 | 3.96 |
df_all_cat.head(3)
entity | continent | |
---|---|---|
0 | Albania | Europe |
1 | Albania | Europe |
2 | Albania | Europe |
df_num_r=df_all_num[['access_to_electricity(%_of_population)', 'renewable_electricity_generating_capacity_per_capita', 'financial_flows_to_developing_countries(us$)', 'electricity_from_nuclear(twh)', 'energy_intensity_level_of_primary_energy(mj/$2017pppgdp)', 'renewables(%equivalent_primary_energy)', 'gdp_per_capita', 'primary_energy_consumption_per_capita(kwh/person)']]
df_num_r= df_num_r.reset_index()
df_num_r = df_num_r.drop(columns=['index'])
df_num_r.head()
access_to_electricity(%_of_population) | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | electricity_from_nuclear(twh) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | renewables(%equivalent_primary_energy) | gdp_per_capita | primary_energy_consumption_per_capita(kwh/person) | |
---|---|---|---|---|---|---|---|---|
0 | 100.0 | 0.0 | 0.0 | 0.0 | 4.13 | 0.0 | 1126.683340 | 9029.4375 |
1 | 100.0 | 0.0 | 0.0 | 0.0 | 3.89 | 0.0 | 1281.659826 | 8635.5320 |
2 | 100.0 | 0.0 | 0.0 | 0.0 | 4.10 | 0.0 | 1425.124219 | 9443.5550 |
3 | 100.0 | 0.0 | 0.0 | 0.0 | 3.80 | 0.0 | 1846.120121 | 10756.6120 |
4 | 100.0 | 0.0 | 0.0 | 0.0 | 3.96 | 0.0 | 2373.581292 | 11586.9510 |
df_all_cat.head()
entity | continent | |
---|---|---|
0 | Albania | Europe |
1 | Albania | Europe |
2 | Albania | Europe |
3 | Albania | Europe |
4 | Albania | Europe |
#X-y split
y=df_num_r['primary_energy_consumption_per_capita(kwh/person)']
X=df_num_r.drop(['primary_energy_consumption_per_capita(kwh/person)'],axis=1)
#Encodiing categorical data
from sklearn.preprocessing import OneHotEncoder
encoder=OneHotEncoder(drop='first').fit(df_all_cat)
encoder=encoder.transform(df_all_cat).toarray()
encoder
array([[1., 0., 0., ..., 1., 0., 0.], [1., 0., 0., ..., 1., 0., 0.], [1., 0., 0., ..., 1., 0., 0.], ..., [0., 0., 0., ..., 0., 0., 1.], [0., 0., 0., ..., 0., 0., 1.], [0., 0., 0., ..., 0., 0., 1.]])
#Normalizing numerical data
from sklearn.preprocessing import StandardScaler
transformer=StandardScaler().fit(X)
X_norm=transformer.transform(X)
X_norm
array([[ 0.43021915, -0.45166585, -0.30391553, ..., -0.20611336, -0.86534058, -0.80663581], [ 0.43021915, -0.45166585, -0.30391553, ..., -0.27625502, -0.86534058, -0.79944867], [ 0.43021915, -0.45166585, -0.30391553, ..., -0.21488107, -0.86534058, -0.79279541], ..., [ 0.41561275, 3.62571671, -0.20282563, ..., -0.59773758, 1.26163592, 0.00851787], [ 0.42291595, 3.62152094, -0.20282563, ..., -0.60358272, 1.26163592, -0.03859265], [ 0.43021915, 3.612449 , -0.20282563, ..., -0.60358272, 1.26163592, -0.14291956]])
X=np.concatenate((X_norm ,encoder),axis=1)
X
array([[ 0.43021915, -0.45166585, -0.30391553, ..., 1. , 0. , 0. ], [ 0.43021915, -0.45166585, -0.30391553, ..., 1. , 0. , 0. ], [ 0.43021915, -0.45166585, -0.30391553, ..., 1. , 0. , 0. ], ..., [ 0.41561275, 3.62571671, -0.20282563, ..., 0. , 0. , 1. ], [ 0.42291595, 3.62152094, -0.20282563, ..., 0. , 0. , 1. ], [ 0.43021915, 3.612449 , -0.20282563, ..., 0. , 0. , 1. ]])
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# Entrenar el modelo RandomForestRegressor
model = RandomForestRegressor(random_state=42) # Puedes ajustar los hiperparámetros aquí
model.fit(X_train, y_train)
# Realizar predicciones en el conjunto de prueba
y_pred = model.predict(X_test)
# Calcular el error (por ejemplo, el error cuadrático medio)
mse = mean_squared_error(y_test, y_pred)
print(f'Error cuadrático medio: {mse}')
# Calcular el error absoluto medio (MAE)
mae = mean_absolute_error(y_test, y_pred)
print(f'Error absoluto medio (MAE): {mae}')
# Calcular el coeficiente de determinación (R-squared)
r2 = r2_score(y_test, y_pred)
print(f'Coeficiente de determinación (R-squared): {r2}')
Error cuadrático medio: 38186658.79575626 Error absoluto medio (MAE): 2700.2914108415234 Coeficiente de determinación (R-squared): 0.9791716055547649
df_all_num.head(3)
year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | landarea(km2) | primary_energy_intensity | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | 2000 | 100.0 | 38.2 | 0.0 | 0.0 | 0.0 | 0.14 | 0.0 | 4.55 | 97.014930 | 9029.4375 | 4.13 | 3170.00000 | 0.0 | 6.946217 | 1126.683340 | 28748.0 | 4.13 |
1 | 2001 | 100.0 | 40.5 | 0.0 | 0.0 | 0.0 | 0.13 | 0.0 | 3.52 | 96.438354 | 8635.5320 | 3.89 | 3230.00000 | 0.0 | 8.293313 | 1281.659826 | 28748.0 | 3.89 |
2 | 2002 | 100.0 | 43.2 | 0.0 | 0.0 | 0.0 | 0.16 | 0.0 | 3.48 | 95.604390 | 9443.5550 | 4.10 | 3759.99999 | 0.0 | 4.536524 | 1425.124219 | 28748.0 | 4.10 |
df_all_cat.head(3)
entity | continent | |
---|---|---|
0 | Albania | Europe |
1 | Albania | Europe |
2 | Albania | Europe |
df_num_k=df_all_num[['access_to_electricity(%_of_population)', 'renewable_electricity_generating_capacity_per_capita', 'financial_flows_to_developing_countries(us$)', 'electricity_from_nuclear(twh)', 'energy_intensity_level_of_primary_energy(mj/$2017pppgdp)', 'renewables(%equivalent_primary_energy)', 'gdp_per_capita', 'primary_energy_consumption_per_capita(kwh/person)']]
df_num_k= df_num_k.reset_index()
df_num_k = df_num_k.drop(columns=['index'])
df_num_k.head()
access_to_electricity(%_of_population) | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | electricity_from_nuclear(twh) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | renewables(%equivalent_primary_energy) | gdp_per_capita | primary_energy_consumption_per_capita(kwh/person) | |
---|---|---|---|---|---|---|---|---|
0 | 100.0 | 0.0 | 0.0 | 0.0 | 4.13 | 0.0 | 1126.683340 | 9029.4375 |
1 | 100.0 | 0.0 | 0.0 | 0.0 | 3.89 | 0.0 | 1281.659826 | 8635.5320 |
2 | 100.0 | 0.0 | 0.0 | 0.0 | 4.10 | 0.0 | 1425.124219 | 9443.5550 |
3 | 100.0 | 0.0 | 0.0 | 0.0 | 3.80 | 0.0 | 1846.120121 | 10756.6120 |
4 | 100.0 | 0.0 | 0.0 | 0.0 | 3.96 | 0.0 | 2373.581292 | 11586.9510 |
#X-y split
y=df_num_k['primary_energy_consumption_per_capita(kwh/person)']
X=df_num_k.drop(['primary_energy_consumption_per_capita(kwh/person)'],axis=1)
#Encodiing categorical data
from sklearn.preprocessing import OneHotEncoder
encoder=OneHotEncoder(drop='first').fit(df_all_cat)
encoder=encoder.transform(df_all_cat).toarray()
encoder
array([[1., 0., 0., ..., 1., 0., 0.], [1., 0., 0., ..., 1., 0., 0.], [1., 0., 0., ..., 1., 0., 0.], ..., [0., 0., 0., ..., 0., 0., 1.], [0., 0., 0., ..., 0., 0., 1.], [0., 0., 0., ..., 0., 0., 1.]])
#Normalizing numerical data
from sklearn.preprocessing import StandardScaler
transformer=StandardScaler().fit(X)
X_norm=transformer.transform(X)
X_norm
array([[ 0.43021915, -0.45166585, -0.30391553, ..., -0.20611336, -0.86534058, -0.80663581], [ 0.43021915, -0.45166585, -0.30391553, ..., -0.27625502, -0.86534058, -0.79944867], [ 0.43021915, -0.45166585, -0.30391553, ..., -0.21488107, -0.86534058, -0.79279541], ..., [ 0.41561275, 3.62571671, -0.20282563, ..., -0.59773758, 1.26163592, 0.00851787], [ 0.42291595, 3.62152094, -0.20282563, ..., -0.60358272, 1.26163592, -0.03859265], [ 0.43021915, 3.612449 , -0.20282563, ..., -0.60358272, 1.26163592, -0.14291956]])
X_r=np.concatenate((X_norm ,encoder),axis=1)
X_r
array([[ 0.43021915, -0.45166585, -0.30391553, ..., 1. , 0. , 0. ], [ 0.43021915, -0.45166585, -0.30391553, ..., 1. , 0. , 0. ], [ 0.43021915, -0.45166585, -0.30391553, ..., 1. , 0. , 0. ], ..., [ 0.41561275, 3.62571671, -0.20282563, ..., 0. , 0. , 1. ], [ 0.42291595, 3.62152094, -0.20282563, ..., 0. , 0. , 1. ], [ 0.43021915, 3.612449 , -0.20282563, ..., 0. , 0. , 1. ]])
from sklearn.neighbors import KNeighborsRegressor
X_train, X_test, y_train, y_test = train_test_split(X_r, y, test_size=0.2, random_state=42)
#Model development
KNN_reg = KNeighborsRegressor(n_neighbors=2)
# Entrenamiento de lo modelo
regressor = KNN_reg.fit(X_train, y_train)
#Predición de modelo
y_pred_knn = regressor.predict(X_test)
plt.figure(figsize=(15, 6))
plt.plot(list(y_test), c="g", label="Actual", linewidth=2)
plt.plot(y_pred_knn, c="r", label="predicción", linewidth=2)
plt.legend(loc='best')
plt.show();
Parece que el Modelo KNN se ajusto bien, los valores pronosticados siguen los valores reales
Finalmente, evaluaré el modelo con diferentes tipos de metricas en el conjunto de prueba (línea de base contra la extrapolación)
#model evaluation
print ("Mean Absolute Error:" , metrics.mean_absolute_error(y_test, y_pred_knn))
print ("Mean Squared Error:" , metrics.mean_squared_error(y_test, y_pred_knn))
print ("Root Mean Squared Error:" , np.sqrt(metrics.mean_squared_error(y_test, y_pred_knn)))
print('Coefficient of determination: %.2f'% r2_score(y_test, y_pred_knn))
Mean Absolute Error: 2434.241779163135 Mean Squared Error: 33791972.84566892 Root Mean Squared Error: 5813.086344246825 Coefficient of determination: 0.98
Elegir un valor K
Sigamos adelante y usemos el método del codo para elegir un buen valor $K$.
from sklearn import neighbors
rmse_val = [] #para almacenar valores rmse para diferentes k
for K in range(20):
K = K+1
model = neighbors.KNeighborsRegressor(n_neighbors = K)
model.fit(X_train, y_train) #fit the model
pred=model.predict(X_test) #hacer predicciones en el equipo de prueba
error = sqrt(mean_squared_error(y_test,pred)) #calculate rmse
rmse_val.append(error) #store rmse values
print('Valor de RMSE para k= ' , K , 'es:', error)
Valor de RMSE para k= 1 es: 6084.615442777433 Valor de RMSE para k= 2 es: 5813.086344246825 Valor de RMSE para k= 3 es: 6069.080572038484 Valor de RMSE para k= 4 es: 6026.956701493579 Valor de RMSE para k= 5 es: 6128.27374103959 Valor de RMSE para k= 6 es: 6230.855395690221 Valor de RMSE para k= 7 es: 6530.754530822095 Valor de RMSE para k= 8 es: 6687.914978937175 Valor de RMSE para k= 9 es: 6969.623170308038 Valor de RMSE para k= 10 es: 7181.545714612904 Valor de RMSE para k= 11 es: 7336.935180334566 Valor de RMSE para k= 12 es: 7667.828954360924 Valor de RMSE para k= 13 es: 8146.976376876385 Valor de RMSE para k= 14 es: 8584.280257657287 Valor de RMSE para k= 15 es: 9216.888099899492 Valor de RMSE para k= 16 es: 9836.75461311659 Valor de RMSE para k= 17 es: 10219.810978095195 Valor de RMSE para k= 18 es: 10709.610485084491 Valor de RMSE para k= 19 es: 11155.015299682436 Valor de RMSE para k= 20 es: 11544.753119777719
plt.figure(figsize=(15,6))
plt.plot(rmse_val ,color="blue", linestyle="dashed", marker="o", markerfacecolor="red", markersize=10)
plt.title('Método del Codo, RMSE vs. K Value', size=20)
plt.xlabel('K')
plt.ylabel('Error Rate')
Text(0, 0.5, 'Error Rate')
# Implementacion de GridserachCV para encontrar el mejor valor de K
from sklearn.model_selection import GridSearchCV
params={'n_neighbors': [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]}
knn= neighbors.KNeighborsRegressor()
model= GridSearchCV(knn, params, cv=5)
model.fit(X_train, y_train)
model.best_params_
{'n_neighbors': 2}
# Function to plot features on world map
def plot_world_map(column_name):
fig = go.Figure()
for year in range(2000, 2021):
# Filter the data for the current year
filtered_df = df_all[df_all['year'] == year]
# Create a choropleth trace for the current year
trace = go.Choropleth(
locations=filtered_df['entity'],
z=filtered_df[column_name],
locationmode='country names',
colorscale='Electric', # Use a different color scale for better contrast
colorbar=dict(title=column_name),
zmin=df[column_name].min(),
zmax=df[column_name].max(),
visible=False # Set the trace to invisible initially
)
# Add the trace to the figure
fig.add_trace(trace)
# Set the first trace to visible
fig.data[0].visible = True
# Create animation steps
steps = []
for i in range(len(fig.data)):
step = dict(
method='update',
args=[{'visible': [False] * len(fig.data)}, # Set all traces to invisible
{'title_text': f'{column_name} Map - {2000 + i}', 'frame': {'duration': 1000, 'redraw': True}}],
label=str(2000 + i) # Set the label for each step
)
step['args'][0]['visible'][i] = True # Set the current trace to visible
steps.append(step)
# Create the slider
sliders = [dict(
active=0,
steps=steps,
currentvalue={"prefix": "Year: ", "font": {"size": 14}}, # Increase font size for slider label
)]
# Update the layout of the figure with increased size and change the template
fig.update_layout(
title_text=f'{column_name}''' '%', # Set the initial title
title_font_size=24, # Increase title font size
title_x=0.5, # Center the title
geo=dict(
showframe=True,
showcoastlines=True,
projection_type='natural earth'
),
sliders=sliders,
height=500, # Set the height of the figure in pixels
width=1000, # Set the width of the figure in pixels
font=dict(family='Arial', size=12), # Customize font family and size for the whole figure
margin=dict(t=80, l=50, r=50, b=50), # Add margin for better layout spacing
template='plotly', # Change the template to 'plotly_dark'
)
# Show the figure
fig.show()
column_name = 'access_to_clean_fuels_for_cooking'
plot_world_map(column_name)
column_name = 'CO2_emissions_value_by_country_kt'
plot_world_map(column_name)
column_name = 'primary_energy_consumption_per_capita(kwh/person)'
plot_world_map(column_name)
column_name = 'electricity_from_renewables(twh)'
plot_world_map(column_name) #no aprece nada en Europa y Afric
column_name = 'renewable_energy_share_in_the_total_final_energy_consumption(%)'
plot_world_map(column_name)
column_name = 'access_to_electricity(%_of_population)'
plot_world_map(column_name) #no aprece nada en Europa y Afr
df_all.head(1)
entity | year | access_to_electricity(%_of_population) | access_to_clean_fuels_for_cooking | renewable_electricity_generating_capacity_per_capita | financial_flows_to_developing_countries(us$) | renewable_energy_share_in_the_total_final_energy_consumption(%) | electricity_from_fossil_fuels(twh) | electricity_from_nuclear(twh) | electricity_from_renewables(twh) | low_carbonelectricity(%electricity) | primary_energy_consumption_per_capita(kwh/person) | energy_intensity_level_of_primary_energy(mj/$2017pppgdp) | CO2_emissions_value_by_country_kt | renewables(%equivalent_primary_energy) | gdp_growth | gdp_per_capita | landarea(km2) | continent | primary_energy_intensity | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | Albania | 2000 | 100.0 | 38.2 | 0.0 | 0.0 | 0.0 | 0.14 | 0.0 | 4.55 | 97.01493 | 9029.4375 | 4.13 | 3170.0 | 0.0 | 6.946217 | 1126.68334 | 28748.0 | Europe | 4.13 |
# Filtra el DataFrame para obtener datos de los años 2010 y 2020
df_filtered = df_all[df_all['year'].isin([2010, 2020])]
# Crea dos gráficos de barras apiladas
plt.figure(figsize=(10, 6)) # Tamaño de la figura
sns.barplot(x='continent', y='access_to_electricity(%_of_population)', hue='year', data=df_filtered, palette=['red', 'blue'])
# Personaliza el gráfico
plt.xlabel('Continente')
plt.ylabel('Acceso a Electricidad (%)')
plt.title('Comparativa de Acceso a Electricidad por Continente (2010 vs. 2020)')
# Agregar etiquetas de datos a las barras
for p in plt.gca().patches:
plt.gca().annotate(f'{p.get_height():.2f}%', (p.get_x() + p.get_width() / 2, p.get_height()), ha='center', va='bottom')
# Muestra el gráfico
plt.show()
# Filtra el DataFrame para obtener datos de los años 2010 y 2020 solo para Asia y Sudamérica
continents_to_plot = ['Asia', 'South America']
df_filtered = df_all[(df_all['year'].isin([2010, 2020])) & (df_all['continent'].isin(continents_to_plot))]
# Crea un gráfico de barras apiladas
plt.figure(figsize=(10, 6)) # Tamaño de la figura
sns.barplot(x='continent', y='access_to_electricity(%_of_population)', hue='year', data=df_filtered, palette=['red', 'blue'])
# Personaliza el gráfico
plt.xlabel('Continente')
plt.ylabel('Acceso a Electricidad (%)')
plt.title('Comparativa de Acceso a Electricidad en Asia y Sudamérica (2010 vs. 2020)')
# Agregar etiquetas de datos a las barras
for p in plt.gca().patches:
plt.gca().annotate(f'{p.get_height():.2f}%', (p.get_x() + p.get_width() / 2, p.get_height()), ha='center', va='bottom')
# Muestra el gráfico
plt.show()