Notebook Version: 2.0
Python Version: Python 3.8+
Required Packages: msticpy, msticnb
Data Sources Required:
Brings together a series of queries and visualizations to help you determine the security state of an Account.
The account can be a Windows or Linux account or an Azure Active Directory/Office 365 account.
The notebook uses the MSTIC notebooklets package to run most of the functionality.
Summarized data is returned when it is run and more detailed information is contained in the returned result
class.
Our broad initial hunting hypothesis is that a we have received account name entity which is suspected to be compromised and is being used malicious manner in internal networks, we will need to hunt from a range of different positions to validate or disprove this hypothesis.
This should complete without errors. If you encounter errors or warnings look at the following notebooks:
If you are running in the Azure Sentinel Notebooks environment (Azure Notebooks or Azure ML) you can run live versions of these notebooks:
You may also need to do some additional configuration to successfully use functions such as Threat Intelligence service lookup and Geo IP lookup.
There are more details about this in the ConfiguringNotebookEnvironment
notebook and in these documents:
from datetime import datetime, timedelta, timezone
REQ_PYTHON_VER = "3.8"
REQ_MSTICPY_VER = "1.8.0"
# %pip install --upgrade msticpy
import msticpy
msticpy.init_notebook(
namespace=globals(),
additional_packages=["msticnb>=1.0"],
verbosity=0,
);
# papermill default parameters
ws_name = "Default"
account_name = ""
account_types = ["All"] # Windows, Linux, Azure, All
end = datetime.now(timezone.utc)
start = end - timedelta(days=2)
workspace_cs = "loganalytics://code().tenant('TENANT_ID').workspace('WORKSPACE_ID')"
e.g.
workspace_cs = "loganalytics://code().tenant('c3de0f06-dcb8-40fb-9d1a-b62faea29d9d').workspace('c62d3dc5-11e6-4e29-aa67-eac88d5e6cf6')"
Then in the Authentication cell replace
the call to qry_prov.connect
with the following:
qry_prov.connect(connect_str=workspace_cs)
The cell should now look like this:
...
# Authentication
qry_prov = QueryProvider(data_environment="MSSentinel")
qry_prov.connect(connect_str=workspace_cs)
...
On successful authentication you should see a popup schema
button.
To find your Workspace Id go to Log Analytics. Look at the workspace properties to find the ID.
print("Configured workspaces: ", ", ".join(msticpy.settings.get_config("AzureSentinel.Workspaces").keys()))
import ipywidgets as widgets
ws_param = widgets.Combobox(
description="Workspace Name",
value=ws_name,
options=list(msticpy.settings.get_config("AzureSentinel.Workspaces").keys())
)
ws_param
from msticpy.common.timespan import TimeSpan
# Authentication
qry_prov = QueryProvider(data_environment="MSSentinel")
qry_prov.connect(WorkspaceConfig(workspace=ws_param.value))
nb_timespan = TimeSpan(start, end)
qry_prov.query_time.timespan = nb_timespan
md("<hr>")
md("Confirm time range to search", "bold")
qry_prov.query_time
If you are having problems, expand the details section below
The notebook is expecting your Azure Sentinel Tenant ID and Workspace ID to be configured in one of the following places:
msticpyconfig.yaml
in the current folder or location specified by MSTICPYCONFIG
environment variable.config.json
in the current folderFor help with setting up your configuration (if this hasn't been done automatically) see the [Getting Started](./A Getting Started Guide For Azure Sentinel ML Notebooks.ipynb) notebook in the root folder of your Azure-Sentinel-Notebooks project.
This imports the msticnb package and the notebooklets classes.
These are needed for the notebook operations
import msticnb as nb
nb.init(query_provider=qry_prov)
pivot.timespan = qry_prov.query_time.timespan
Type the account name that you want to search for and the time bounds over which you want to search.
You can specify the account as:
alice
)alice@contoso.com
)mydomain\alice
In the second two cases the domain qualifier will be stripped off before the search. The search is not case sensitive and will match full substrings. E.g. bob
will match domain\bob
and bob@contoso.com
but not bobg
or bo
.
account_txt = nbwidgets.GetText(prompt='Enter the Account name to search for:', value=account_name)
display(account_txt)
md("<hr>")
You can opt to search all data types or just a subset. For example, if you know the account activity that you are interested in is only Windows host activity, you can select "Windows".
The default is All Data but other options are:
from msticnb.nb.azsent.account.account_summary import AccountType
acct_types = [item.name for item in AccountType]
import ipywidgets as widgets
acct_types_select = widgets.SelectMultiple(
description="Select Account types to search",
options=acct_types,
value=account_types or acct_types,
**WIDGET_DEFAULTS,
)
acct_types_select
run
method¶The notebooklet will search Azure, Windows host and Linux host data, searching for account matches.
Note: Different result properties will populated for different account types.
It will display a summary of the information retrieved as it is running. You can find information on accessing the full data later in the notebook.
Note: If more than one matching account name is found, all matches will be shown.
You can select each of these matching accounts to view more details about the account.
Once selected, you can retrieve more detailed information about that account.
Account = entities.Account
acc_result = Account.nblt.account_summary(
value=account_txt.value,
account_types=acct_types_select.value
)
In this case you do not have direct access to the methods
of the "AccountSummary".
However, all methods and properties
of the notebooklet class are accessible via the results class.
acct_nb.get_additional_data()
is equivalent to
acc_result.get_additional_data()
acct_nb = nb.nblts.azsent.account.AccountSummary()
md(
"",
"large, bold"
)
acc_result = acct_nb.run(
value=account_txt.value,
timespan=qry_prov.query_time.timespan,
account_types=acct_types_select.value,
)
if len(acc_result.account_selector.options) > 1:
md_warn("Multiple matches found for account. Running on first listed account")
The result returned from the last cell has a number of properties and methods that you can use to retrieve and view further information.
The main one for this notebooklet is get_additional_data
.
Depending on the account type (Azure, Windows or Linux), it will retrieve more detailed
data about recent activity
acc_result.get_additional_data()
If there are any alerts referencing this account name they can be viewed
by calling the acc_result.browse_alerts()
function.
acc_result.notebooklet.browse_alerts()
You can use a simple view to make it easier to see the details of individual events by calling the "view_events" method.
You need to supply the name of the result attribute that you want to view plus one or more summary columns (as a list of strings).
data_source = nbwidgets.SelectItem(
description="Available data properties\n",
item_list=acc_result.data_properties()
)
data_source
Running the following cell will use the data property selected above to browse through the data (if any).
acc_result.view_events(
attrib=data_source.value, # the result attribute to view
# summary_cols controls which cols are use to create the summary
# for the select list
summary_cols = list(getattr(acc_result, data_source.value).columns)[:3]
# Add specific columns here to customize the list items
# summary_cols=["SourceSystem", "Operation", "IPAddress"]
)
You can also access the DataFrames properties directly
acc_result.ip_all_data
You can pass a DataFrame to result.view_events()
instead of an attribute name.
This means that you can apply sorting or filtering of the data before viewing it. Here is an example sorting by IPAddress.
acc_result.view_events(
data=acc_result.azure_activity.sort_values("IPAddress"),
summary_cols=["SourceSystem", "Operation", "IPAddress"]
)
These are static properties - usually DataFrames or visualizations. You can access each of these to see or manipulate the retrieved data.
To see help on the available attributes type:
>>> help(acc_result)
To see the available methods type:
>>> acc_result.list_methods()
Note, for the AccountSummary notebooklet, the two main data retrieval methods are:
- run
- get_additional_data
There are several other methods that allow you to view individual plots or subsets of the data (such as alerts).
To view help on a specific method type:
>>> help(acc_result.method_name)
To run a method
acc_result.display_alert_timeline()
acc_result.list_methods()
You can view all of the data in the results class by "running" it in a cell
Note: This produces a lot of output.
Due to the way Jupyter display Javascript objects the plots may appear out of order.
acc_result
Most of the properties of the results class are pandas DataFrames - you can use these directly for further analysis. Other property types include entities and visualizations.
The DataFrames displayed by running the result object are truncated to the first five rows.
You can also access individual data properties of the result as follows:
result.data_property
acc_result.data_properties()
You can run a pivot function on the account summary results to get additional context on the data.
Here is an example of looking up Whois information for Azure IPAddress requests.
whois_df = (
acc_result # the results object
.azure_activity[["IPAddress"]] # the property and the column we want
.drop_duplicates() # drop duplicates
.mp_pivot.run( # run the pivot function IpAddress 'whois' function
IpAddress.util.whois, column="IPAddress"
)
)
whois_df
You may want to drill down on other entities in the Account data. You can use methods of the IpAddress or Host entities, for example, to look at these in more detail.
Run the ip_address_summary notebooklet pivot
IpAddress = entities.IpAddress
ip_result = IpAddress.nblt.ip_address_summary("157.56.162.53")
View the TI results
ip_result.browse_ti_results()