#!/usr/bin/env python
# coding: utf-8

# # Inspecting Data
# 
# This example shows how to inspect the data of a loaded test system.

# In[1]:


import andes
from andes.utils.paths import get_case

andes.config_logger()


# To show all the rows and columns, change the pandas configuration with

# In[2]:


import pandas as pd

pd.options.display.max_columns = None
pd.options.display.max_rows = None


# Let's load the Kundur's system.

# ## Load System from an ANDES XLSX File
# 
# The ANDES xlsx file is the best supported format. Other formats can be converted to the xlsx format.
# 
# See the link below for more about format conversion.
# https://github.com/cuihantao/andes/blob/master/README.md#format-converter

# As previously shown, test cases can be loaded with ``andes.run()``:

# In[3]:


ss = andes.run(get_case('kundur/kundur_full.xlsx'))


# Alternatively, one can load a test case _without setting up_ using `andes.load(..., setup=False)`. Note that `setup=False` option.
# It is useful to apply parameter changes to an existing test case.

# In[4]:


ss = andes.load(get_case('kundur/kundur_full.xlsx'), setup=False)


# For example, we can toggle the connectivity status `u` of `Line_1` to `0` using

# In[5]:


ss.Line.alter('u', 'Line_1', 0)


# When done, remember to set up the system before running calculation routines:

# In[6]:


ss.setup()

ss.PFlow.run()


# After setting up the system, adding or removing devices are not yet allowed.

# ## Load System from PSS/E RAW and DYR Files
# 
# ANDES supports loading systems from PSS/E RAW and DYR files.
# 
# The PSS/E v32 raw format is best supported.
# 
# Note that this feature is experimental. We try out best to support this format, but the compatibility is not guaranteed.

# In[7]:


raw_path = get_case('kundur/kundur.raw')
dyr_path = get_case('kundur/kundur_full.dyr')


# The raw file is passed to the positional argument, whereas the dyr file is passed to `addfile`.

# In[8]:


ss = andes.run(raw_path, addfile=dyr_path)


# ### Attributes for storing values

# Parameters are stored as attributes of the model. For example, `ss.GENROU.M`, the machine starting time constant (`2H`), is stored in `ss.GENROU.M`.

# In[9]:


ss.GENROU.M


# It is an instance of `NumParam`, which contains fields `v` for the values after converting to system-base per unit values.

# In[10]:


ss.GENROU.M.v


# And field `vin` is for the original input data.

# In[11]:


ss.GENROU.M.vin


# ### Tabulated view

# ANDES provides tabulated **view** of model parameters by using DataFrame. Each model object has an attribute called `cache` for caching the parameter dataframes.

# The original parameters from the input file are stored in `cache.df_in` of the model object. For `GENROU`, do

# In[12]:


ss.GENROU.cache.df_in


# Parameters will be **converted** to per-unit in the system base after loading. This process have been done if `andes.run` is used for loading the data file.
# 
# To inspect the converted parameters, check the `cache.df` parameter.

# In[13]:


ss.GENROU.cache.df


# One will notice the converted parameters such as `M`, `xl`, and all other impedances.

# **It is very important to notice that `cache.df` and `cache.df_in` are both views. Altering data in these views will NOT alter the underlying parameter values.**
# 
# To alter values, see the example below.

# ### Altering parameters

# Parameters can be altered by calling the `alter` method on a model instance. 
# 
# We first look up the original value through `get`. 
# 
# Either `v` or `vin` can be passed to argument `attr` to retrieve the converted or the original data. Here we are retrieving the original input data. If `attr` is not provided, `get` returns the value after per-unit conversion, which is the value used for calculation, by default. 

# In[14]:


ss.GENROU.get("M", "GENROU_1", attr='vin')


# To change the `M` of `GENROU_1` to `10`, do

# In[15]:


ss.GENROU.alter("M", "GENROU_1", 10)


# The value set through `alter` is always the data before per-unit conversion - just like it should have been in an input file. ANDES will perform the conversion and set `vin` and `v` correctly.

# Parameters altered through `Model.alter()` can be saved as a new system using

# In[16]:


andes.io.xlsx.write(ss, 'new_system.xlsx', overwrite=True)


# ### Refreshing the view

# As mentioned, `cache.df` and `cache.df_in` are *cached* views and will not be automatically updated for inspection.
# 
# This is generally not an issue if one performs the simulation after altering data. However, if one needs to inspect the data again, `cache.refresh()` needs to be called manually.
# 

# In[17]:


ss.GENROU.cache.refresh()


# In[18]:


ss.GENROU.cache.df_in


# ## Variables

# ### Snapshots

# One might also want to check the variable values in a similar way to that for a parameter. Certainly, a variable has a `v` attribute which stores values. 
# 
# **It is important to note that `v` only holds the values at the last program state.** Such program state could be the solution of power flow, the initialization of time-domain simulation, or the end of a simulation disturbances. 

# Since we have only ran power flow for ``ss``, ``ss.Bus.v.v`` are the voltage magnitude solutions, where the first `v` is for "voltage", and the second `v` is the first `v`'s value attribute.

# In[19]:


ss.Bus.v.v


# Variables hold more than values. They have an attribute `a` for the addresses indexing into the corresponding type of array.

# There are two system-level arrays, `ss.dae.x` and `ss.dae.y` for the right-hand-side of the differential and algebraic equations, respectively. 

# In[20]:


type(ss.Bus.v)


# `ss.Bus.v` is an algebraic variable, thus `ss.Bus.v.a` holds the indices into ``ss.dae.g``.

# In[21]:


ss.dae.y[ss.Bus.v.a]


# We can see that these two values are the same.

# ### Time series

# After a time-domain simulation, the time series of the variables can be retrieved through `ss.dae.ts`. Let's first run a simulation.

# In[22]:


ss.TDS.run()


# In[23]:


ss.dae.ts


# `ss.dae.ts` has four commonly used attributes: `t` for time stamps, `xy` for variables (differential and then algebraic), `z` for discontinuous states, and `df` for the dataframe of all.
# 
# - Each point in `ss.dae.ts.t` correspond to a row in `ss.dae.ts.xy`.
# - Each column in `ss.dae.ts.xy` correspond to a variable, whose name can be located in `ss.dae.xy_name`, for all timestamps.
# - `z` is not stored by default unless one enables it before simulation by setting `ss.TDS.config.store_z = 1`. 
# - `df` is not built by default but can be manually triggered after simulation by calling `ss.dae.ts.unpack(df=True)`

# The following are some statistics of the shapes of arrays:

# In[24]:


ss.dae.ts.t.shape


# In[25]:


ss.dae.ts.xy.shape


# In[26]:


len(ss.dae.xy_name)


# ## Cleanup

# In[27]:


get_ipython().system('andes misc -C')


# In[28]:


get_ipython().system('rm new_system.xlsx')