#!/usr/bin/env python
# coding: utf-8
# # Entity Explorer - Account
#
# Notebook Details...
#
# **Notebook Version:** 2.0
# **Python Version:** Python 3.8+
# **Required Packages**: msticpy, msticnb
#
# **Data Sources Required**:
# - Sentinel - SecurityAlert, SecurityEvent, HuntingBookmark, Syslog, AAD SigninLogs, AzureActivity, OfficeActivity, ThreatIndicator
# - (Optional) - VirusTotal, AlienVault OTX, IBM XForce, Open Page Rank, (all require accounts and API keys)
#
#
# Brings together a series of queries and visualizations to help you determine the security state of an Account.
#
# The account can be a Windows or Linux account or an Azure Active Directory/Office 365 account.
#
# The notebook uses the [MSTIC notebooklets](https://msticnb.readthedocs.io) package to run most of the functionality.
# Summarized data is returned when it is run and more detailed information is contained in the returned `result` class.
#
#
#
#
#
# Contents
#
#
#
#
#
# ## Hunting Hypothesis
# Our broad initial hunting hypothesis is that a we have received account name entity which is suspected to be compromised and is being used malicious manner in internal networks, we will need to hunt from a range of different positions to validate or disprove this hypothesis.
# ---
# # Notebook initialization
# This should complete without errors. If you encounter errors or warnings look at the following notebooks:
#
# - Getting Started Notebook
# - [TroubleShootingNotebooks](https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/TroubleShootingNotebooks.ipynb)
# - [ConfiguringNotebookEnvironment](https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/ConfiguringNotebookEnvironment.ipynb)
#
#
# Details...
# The next cell:
# - Checks for the correct Python version
# - Checks versions and optionally installs required packages
# - Imports the required packages into the notebook
# - Sets a number of configuration options.
#
# If you are running in the Azure Sentinel Notebooks environment (Azure Notebooks or Azure ML) you can run live versions of these notebooks:
# - [Getting Started](./A Getting Started Guide For Azure Sentinel ML Notebooks.ipynb)
# - [Run TroubleShootingNotebooks](./TroubleShootingNotebooks.ipynb)
# - [Run ConfiguringNotebookEnvironment](./ConfiguringNotebookEnvironment.ipynb)
#
# You may also need to do some additional configuration to successfully use functions such as Threat Intelligence service lookup and Geo IP lookup.
# There are more details about this in the `ConfiguringNotebookEnvironment` notebook and in these documents:
# - [msticpy configuration](https://msticpy.readthedocs.io/en/latest/getting_started/msticpyconfig.html)
# - [Threat intelligence provider configuration](https://msticpy.readthedocs.io/en/latest/data_acquisition/TIProviders.html#configuration-file)
#
# In[ ]:
from datetime import datetime, timedelta, timezone
REQ_PYTHON_VER = "3.8"
REQ_MSTICPY_VER = "1.8.0"
# %pip install --upgrade msticpy
import msticpy
msticpy.init_notebook(
namespace=globals(),
additional_packages=["msticnb>=1.0"],
verbosity=0,
);
# In[ ]:
# papermill default parameters
ws_name = "Default"
account_name = ""
account_types = ["All"] # Windows, Linux, Azure, All
end = datetime.now(timezone.utc)
start = end - timedelta(days=2)
# ### Get Workspace and Authenticate
#
#
# Authentication help...
# If you want to use a workspace other than one you have defined in your
# msticpyconfig.yaml create a connection string with your AAD TENANT_ID and
# your WORKSPACE_ID (these should both be quoted UUID strings).
#
# ```python
# workspace_cs = "loganalytics://code().tenant('TENANT_ID').workspace('WORKSPACE_ID')"
# ```
# e.g.
# ```python
# workspace_cs = "loganalytics://code().tenant('c3de0f06-dcb8-40fb-9d1a-b62faea29d9d').workspace('c62d3dc5-11e6-4e29-aa67-eac88d5e6cf6')"
# ```
# Then in the Authentication cell replace
# the call to `qry_prov.connect` with the following:
# ```python
# qry_prov.connect(connect_str=workspace_cs)
# ```
# The cell should now look like this:
#
# ```python
# ...
# # Authentication
# qry_prov = QueryProvider(data_environment="MSSentinel")
# qry_prov.connect(connect_str=workspace_cs)
# ...
# ```
#
# On successful authentication you should see a ```popup schema``` button.
# To find your Workspace Id go to [Log Analytics](https://ms.portal.azure.com/#blade/HubsExtension/Resources/resourceType/Microsoft.OperationalInsights%2Fworkspaces). Look at the workspace properties to find the ID.
#
# In[ ]:
print("Configured workspaces: ", ", ".join(msticpy.settings.get_config("AzureSentinel.Workspaces").keys()))
import ipywidgets as widgets
ws_param = widgets.Combobox(
description="Workspace Name",
value=ws_name,
options=list(msticpy.settings.get_config("AzureSentinel.Workspaces").keys())
)
ws_param
# In[ ]:
from msticpy.common.timespan import TimeSpan
# Authentication
qry_prov = QueryProvider(data_environment="MSSentinel")
qry_prov.connect(WorkspaceConfig(workspace=ws_param.value))
nb_timespan = TimeSpan(start, end)
qry_prov.query_time.timespan = nb_timespan
md("
")
md("Confirm time range to search", "bold")
qry_prov.query_time
# #### Authentication and Configuration problems?
#
# If you are having problems, expand the details section below
#
#
# Click for details about configuring your authentication parameters
#
#
# The notebook is expecting your Azure Sentinel Tenant ID and Workspace ID to be configured in one of the following places:
# - `msticpyconfig.yaml` in the current folder or location specified by `MSTICPYCONFIG` environment variable.
# - `config.json` in the current folder
#
# For help with setting up your configuration (if this hasn't been done automatically) see the [Getting Started](./A Getting Started Guide For Azure Sentinel ML Notebooks.ipynb) notebook in the root folder of your Azure-Sentinel-Notebooks project.
#
# ## Import and initialize notebooklets
#
# This imports the **msticnb** package and the notebooklets classes.
#
# These are needed for the notebook operations
# In[ ]:
import msticnb as nb
nb.init(query_provider=qry_prov)
pivot.timespan = qry_prov.query_time.timespan
# # Enter account name and query time window
# Type the account name that you want to search for and the time bounds over which you want to search.
#
# You can specify the account as:
#
# - a simple user name (e.g. `alice`)
# - a user principal name (`alice@contoso.com`)
# - a qualified windows user name `mydomain\alice`
#
# In the second two cases the domain qualifier will be stripped off before the search. The search is not case sensitive and will match full substrings. E.g. `bob` will match `domain\bob` and `bob@contoso.com` but not `bobg` or `bo`.
# In[ ]:
account_txt = nbwidgets.GetText(prompt='Enter the Account name to search for:', value=account_name)
display(account_txt)
md("
")
# You can opt to search all data types or just a subset.
# For example, if you know the account activity that you are interested in is only Windows host activity, you can select "Windows".
#
# The default is All Data but other options are:
# - Azure (includes Office365 activity, Active Directory Signin logs and Azure audit activity)
# - Office365 - only Office365 activity
# - AzureActiveDirectory - only signin logs
# - Azure - only Azure resource/audit activity
# - Windows - only Windows host event logs
# - Linux - only Linux host syslog
# In[ ]:
from msticnb.nb.azsent.account.account_summary import AccountType
acct_types = [item.name for item in AccountType]
import ipywidgets as widgets
acct_types_select = widgets.SelectMultiple(
description="Select Account types to search",
options=acct_types,
value=account_types or acct_types,
**WIDGET_DEFAULTS,
)
acct_types_select
# # Run the main *account_summary* notebooklet `run` method
#
# The notebooklet will search Azure, Windows host and Linux host data,
# searching for account matches.
#
# **Note**: Different result properties will populated for different account types.
#
# It will display a *summary* of the information retrieved as it is running.
# You can find information on accessing the full data later in the notebook.
#
# **Note:** If more than one matching account name is found, all matches will be shown.
# You can select each of these matching accounts to view more details about the account.
# Once selected, you can retrieve more detailed information about that account.
#
#
# Running the notebooklet as a Pivot Function
# This can also be run as a pivot function from the Account entity.
# The pivot function `account_summary` is in the `nblt` container
# of the `Account` entity.
#
# ```python
# Account = entities.Account
# acc_result = Account.nblt.account_summary(
# value=account_txt.value,
# account_types=acct_types_select.value
# )
# ```
#
# In this case you do not have direct access to the methods
# of the "AccountSummary".
# However, all methods and properties
# of the notebooklet class are accessible via the results class.
# ```python
# acct_nb.get_additional_data()
# ```
# is equivalent to
# ```python
# acc_result.get_additional_data()
# ```
#
# In[ ]:
acct_nb = nb.nblts.azsent.account.AccountSummary()
md(
"",
"large, bold"
)
acc_result = acct_nb.run(
value=account_txt.value,
timespan=qry_prov.query_time.timespan,
account_types=acct_types_select.value,
)
# In[ ]:
if len(acc_result.account_selector.options) > 1:
md_warn("Multiple matches found for account. Running on first listed account")
# # Retrieve additional data for the selected account
#
# The result returned from the last cell has a number of properties and methods that you can use
# to retrieve and view further information.
#
# The main one for this notebooklet is `get_additional_data`.
# Depending on the account type (Azure, Windows or Linux), it will retrieve more detailed
# data about recent activity
# In[ ]:
acc_result.get_additional_data()
# # Browse alerts
# If there are any alerts referencing this account name they can be viewed
# by calling the `acc_result.browse_alerts()` function.
# In[ ]:
acc_result.notebooklet.browse_alerts()
# # Appendix - Additional properties from the Notebooklet result
#
# ---
# ## Browse other event data
#
# You can use a simple view to make it easier to see the details of individual events
# by calling the "view_events" method.
#
# You need to supply the name of the result attribute that you want to view plus
# one or more summary columns (as a list of strings).
# In[ ]:
data_source = nbwidgets.SelectItem(
description="Available data properties\n",
item_list=acc_result.data_properties()
)
data_source
# Running the following cell will use the data property selected above
# to browse through the data (if any).
# In[ ]:
acc_result.view_events(
attrib=data_source.value, # the result attribute to view
# summary_cols controls which cols are use to create the summary
# for the select list
summary_cols = list(getattr(acc_result, data_source.value).columns)[:3]
# Add specific columns here to customize the list items
# summary_cols=["SourceSystem", "Operation", "IPAddress"]
)
# You can also access the DataFrames properties directly
# In[ ]:
acc_result.ip_all_data
# ## Browse events with alternative sorting
# You can pass a DataFrame to *result*`.view_events()` instead of an attribute name.
#
# This means that you can apply sorting or filtering of the data before viewing it.
# Here is an example sorting by IPAddress.
#
# ```python
# acc_result.view_events(
# data=acc_result.azure_activity.sort_values("IPAddress"),
# summary_cols=["SourceSystem", "Operation", "IPAddress"]
# )
# ```
# ## Additional properties and methods of the result object
#
# These are static properties - usually DataFrames or visualizations.
# You can access each of these to see or manipulate the retrieved data.
#
# To see help on the available attributes type:
# ```python
# >>> help(acc_result)
# ```
# To see the available methods type:
# ```python
# >>> acc_result.list_methods()
# ```
# > Note, for the AccountSummary notebooklet, the two main data retrieval methods are:
# > - run
# > - get_additional_data
# > There are several other methods that allow you to view individual plots
# > or subsets of the data (such as alerts).
#
# To view help on a specific method type:
# ```python
# >>> help(acc_result.method_name)
# ```
#
# To run a method
# ```python
# acc_result.display_alert_timeline()
# ```
# In[ ]:
acc_result.list_methods()
# ---
#
# ## Viewing the Result class
# You can view all of the data in the results class by "running" it in a cell
#
# > Note: This produces a lot of output.
# > Due to the way Jupyter display Javascript objects the plots may
# > appear out of order.
#
# ```ipython
# acc_result
# ```
#
# Most of the properties of the results class are pandas DataFrames -
# you can use these directly for further analysis. Other property types
# include entities and visualizations.
#
# The DataFrames displayed by running the result object are truncated
# to the first five rows.
#
# You can also access individual data properties of the result as follows:
# ```ipython
# result.data_property
# ```
# In[ ]:
acc_result.data_properties()
# ---
#
# ## Using Pivots to get more context information
#
# You can run a pivot function on the account summary results
# to get additional context on the data.
#
# Here is an example of looking up Whois information for Azure IPAddress requests.
#
# ```python
# whois_df = (
# acc_result # the results object
# .azure_activity[["IPAddress"]] # the property and the column we want
# .drop_duplicates() # drop duplicates
# .mp_pivot.run( # run the pivot function IpAddress 'whois' function
# IpAddress.util.whois, column="IPAddress"
# )
# )
# whois_df
# ```
# ---
#
# ## Use other notebooklets and pivots functions to drill down on other entities
#
# You may want to drill down on other entities in the Account data.
# You can use methods of the IpAddress or Host entities, for example,
# to look at these in more detail.
#
# Run the ip_address_summary notebooklet pivot
# ```python
# IpAddress = entities.IpAddress
# ip_result = IpAddress.nblt.ip_address_summary("157.56.162.53")
# ```
#
# View the TI results
# ```python
# ip_result.browse_ti_results()
# ```
# ---
#
# # More information:
#
# ## Notebooklets and Pivots
# [Notebooklets](https://msticnb.readthedocs.io/en/latest/)
#
# [Pivot functions](https://msticpy.readthedocs.io/en/latest/data_analysis/PivotFunctions.html)
#
# ## Notebook/MSTICPy configuration
# [Getting Started](https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/A%20Getting%20Started%20Guide%20For%20Azure%20Sentinel%20ML%20Notebooks.ipynb)
# [MSTICPy Configuration guide](https://msticpy.readthedocs.io/en/latest/getting_started/msticpyconfig.html)
#
# [ConfigureNotebookEnvironment notebook](https://github.com/Azure/Azure-Sentinel-Notebooks/blob/master/ConfiguringNotebookEnvironment.ipynb)