Entity Explorer: IP Address
Notebook Version: 2.0
Python Version: Python 3.8+
Required Packages: msticpy, msticnb
Data Sources Required:
MS Sentinel (mostly optional)
External
This notebook brings together a series of queries and visualizations to help you assess the security state of an IP address.
It works with both internal addresses and public addresses.
The notebook uses the MSTIC notebooklets package to run most of the functionality.
Summarized data is returned when it is run and more detailed information is contained in the returned result
class.
Our broad initial hunting hypothesis is that a we have received IP address entity which is suspected to be compromized internal host or external public address to whom internal hosts are communicating in malicious manner, we will need to hunt from a range of different positions to validate or disprove this hypothesis.
This should complete without errors. If you encounter errors or warnings look at the following notebooks:
If you are running in the Azure Sentinel Notebooks environment (Azure Notebooks or Azure ML) you can run live versions of these notebooks:
You may also need to do some additional configuration to successfully use functions such as Threat Intelligence service lookup and Geo IP lookup.
There are more details about this in the ConfiguringNotebookEnvironment
notebook and in these documents:
from datetime import datetime, timedelta, timezone
REQ_PYTHON_VER="3.8"
REQ_MSTICPY_VER="1.8.0"
# You may need to manually install msticpy with
# %pip install msticpy[azsentinel]
import msticpy
msticpy.init_notebook(
namespace=globals(),
additional_packages=["msticnb>=1.0"],
verbosity=0,
);
# papermill default parameters
ws_name = "Default"
ip_address = ""
end = datetime.now(timezone.utc)
start = end - timedelta(days=2)
workspace_cs = "loganalytics://code().tenant('TENANT_ID').workspace('WORKSPACE_ID')"
e.g.
workspace_cs = "loganalytics://code().tenant('c3de0f06-dcb8-40fb-9d1a-b62faea29d9d').workspace('c62d3dc5-11e6-4e29-aa67-eac88d5e6cf6')"
Then in the Authentication cell replace
the call to qry_prov.connect
with the following:
qry_prov.connect(connect_str=workspace_cs)
The cell should now look like this:
...
# Authentication
qry_prov = QueryProvider(data_environment="MSSentinel")
qry_prov.connect(connect_str=workspace_cs)
...
On successful authentication you should see a popup schema
button.
To find your Workspace Id go to Log Analytics. Look at the workspace properties to find the ID.
Note: in VSCode (bug) the options may display in the VSCode status bar (bottom left)
print("Configured workspaces: ", ", ".join(msticpy.settings.get_config("AzureSentinel.Workspaces").keys()))
import ipywidgets as widgets
ws_param = widgets.Combobox(
description="Workspace Name",
value=ws_name,
options=list(msticpy.settings.get_config("AzureSentinel.Workspaces").keys())
)
ws_param
from msticpy.common.timespan import TimeSpan
# Authentication
qry_prov = QueryProvider(data_environment="MSSentinel")
qry_prov.connect(WorkspaceConfig(workspace=ws_param.value))
nb_timespan = TimeSpan(start, end)
qry_prov.query_time.timespan = nb_timespan
md("<hr>")
md("Confirm time range to search", "bold")
qry_prov.query_time
If you are having problems, expand the details section below
The notebook is expecting your Azure Sentinel Tenant ID and Workspace ID to be configured in one of the following places:
msticpyconfig.yaml
in the current folder or location specified by MSTICPYCONFIG
environment variable.config.json
in the current folderFor help with setting up your configuration (if this hasn't been done automatically) see the [Getting Started](./A Getting Started Guide For Azure Sentinel ML Notebooks.ipynb) notebook in the root folder of your Azure-Sentinel-Notebooks project.
This imports the msticnb package and the notebooklets classes.
These are needed for the notebook operations
import msticnb as nb
nb.init(query_provider=qry_prov)
pivot.timespan = qry_prov.query_time.timespan
Type the IP address you want to search for and the time bounds over which search.
You can specify the IP address value in the widget e.g. 192.168.1.1
ipaddr_text = nbwidgets.GetText(prompt='Enter the IP Address to search for:', value=ip_address)
display(ipaddr_text)
md("<hr>")
run
method to retrieve IP information¶The notebooklet will query a variety of Sentinel tables and external data sources including:
A summary of the data is displayed as it runs. You can run view the notebooklet result class to view the whole content or use individual data properties (DataFrames) to view detailed information.
See later in the notebook for how to access these.
Include additional options by prefixing the option with a "+"
See the code in the next cell for an example.
To remove an option, include in the list prefixed with a "-"
Note: this can also be run as a pivot function from the IpAddress entity.
The pivot functionip_summary
is in thenblt
container of theAccount
entity.
IpAddress = entities.IpAddress
ip_address = ipaddr_text.value.strip()
ip_result = IpAddress.nblt.ip_address_summary(
value=ip_address,
options=["+az_activity", "+office_365", "+common_security"]
)
ip_addr_nb = nb.nblts.azsent.network.IpAddressSummary()
md(
"Note: Different result properties are populated depending on the IP type",
"large, bold"
)
ip_result = ip_addr_nb.run(
value=ipaddr_text.value.strip(),
timespan=qry_prov.query_time.timespan,
options=["+az_activity", "+office_365", "+common_security"],
)
ip_result.browse_alerts()
ip_result.browse_ti_results()
Show the location of the IP address on a map
folium_map = FoliumMap(zoom_start=8)
icon_props = {"color": "green"}
folium_map.add_ip_cluster(ip_entities=[ip_result.ip_entity], **icon_props)
folium_map.center_map()
display(folium_map)
The notebooklet result has a number of data properties
ip_result.data_properties()
ip_result.list_methods()
These are static properties - usually DataFrames or visualizations. You can access each of these to see or manipulate the retrieved data.
To see help on the available attributes type:
>>> help(ip_result)
To see the available methods type:
>>> ip_result.list_methods()
Note, for the IP Summary notebooklet, the main data retrieval method is:
- run
There are several other methods that allow you to view individual plots or subsets of the data (such as alerts).
To view help on a specific method type:
>>> help(ip_result.method_name())
ip_result.passive_dns.T
You can view all of the data in the results class by "running" it in a cell
Note: This produces a lot of output.
Due to the way Jupyter display Javascript objects the plots may appear out of order.
ip_result
Most of the properties of the results class are pandas DataFrames - you can use these directly for further analysis. Other property types include entities and visualizations.
The DataFrames displayed by running the result object are truncated to the first five rows.
You can also access individual data properties of the result as follows:
result.data_property
ip_result
You can run a pivot function on the summary results to get additional context on the data.
Here is an example of looking up Whois information for Azure IPAddress requests.
whois_df = (
ip_result # the results object
.azure_activity_summary[["IPAddress"]] # the property and the column we want
.drop_duplicates() # drop duplicates
.mp_pivot.run( # run the pivot function IpAddress 'whois' function
IpAddress.util.whois, column="IPAddress"
)
)
whois_df
You may want to drill down on other entities in the data. You can use methods of the IpAddress or Host entities, for example, to look at these in more detail.
Run the ip_address_summary notebooklet pivot
acc_result = Account.nblt.account_summary("user@my-aad.com")