This notebook shows you how you can use the power of Microsoft 365 Defender, Microsoft Sentinel, & the Microsoft Graph in order to find and investigate malicious links shared with users via Microsoft Teams.
Pre-requisites:
# You may need to manually install msticpy with
# %pip install msticpy[azsentinel]
import msticpy as mp
mp.init_notebook(
namespace=globals(),
verbosity=0,
);
sent_provider = mp.QueryProvider("MicrosoftSentinel")
sent_provider.connect()
graph_provider = mp.QueryProvider("SecurityGraph", delegated_auth=True)
graph_provider.connect()
Investigating all links shared via Teams is going to provide a very large set of data. We can start our investigation from a more limited set of data but looking for events were SmartScreen was triggered after a user opened a link from Teams. To do that we need to look at Microsoft Defender for Endpoint data for SmartScreen events where the opening process was teams.
Note: You could also use the new UrlClickEvent dataset in order to look for links shared via Teams.
# Get smart screen events triggered by Teams:
query = """DeviceEvents
| where TimeGenerated > ago(30d)
| where ActionType == "SmartScreenUrlWarning"
| join (DeviceEvents | where ActionType == "BrowserLaunchedToOpenUrl" | extend OpeningProcess = InitiatingProcessFileName) on DeviceId, RemoteUrl
| extend TeamsUser = InitiatingProcessAccountUpn1
| where OpeningProcess =~ "teams.exe"
| project-reorder DeviceName, RemoteUrl, OpeningProcess, TeamsUser"""
smartscreen_df = sent_provider.exec_query(query)
smartscreen_df.head()
Once we have a these events we can search scope our investigation by focussing on links shared in the Teams Channels that the users associated with these SmartScreen events are members of.
# Get Teams a User is part of by querying the Graph.
users = smartscreen_df["TeamsUser"].unique()
team_membership = []
for user in users:
team_membership_df = graph_provider.exec_query(f"/users/{user}/joinedTeams")
teams_ids = [team["id"] for team in team_membership_df["value"].iloc[0]]
teams_names = [team["id"] for team in team_membership_df["value"].iloc[0]]
teams = pd.DataFrame({"ID": teams_ids, "Name": teams_names})
team_membership.append(teams)
teams_df = pd.concat(team_membership)
md("Teams to investigate:")
display(teams_df)
Now that we have a set of Teams to investigate we can use OfficeActivity logs to find all the messages that have a URL in them.
# Get links in those teams
msgs_query = f"""
let teams = dynamic({list(teams['ID'].unique())});
OfficeActivity
| where TimeGenerated > ago(30d)
| where OfficeWorkload =~ "MicrosoftTeams"
| where Operation in ("MessageCreatedHasLink", "MessageUpdatedHasLink")
| where AADGroupId in (teams)
| project MessageId, AADGroupId, ChannelGuid"""
msgs_df = sent_provider.exec_query(msgs_query)
md("Messages containing URLs from these Teams:")
display(msgs_df)
The OfficeActivity logs don't contain details of the messages themselves, just a message ID. To get the message content, and the include URLs we need to query the Microsoft Graph.
graph_provider.api_ver = "beta"
link_messages = []
for item in msgs_df.iterrows():
df = graph_provider.exec_query(
f"/teams/{item[1]['AADGroupId']}/channels/{item[1]['ChannelGuid']}/messages/{item[1]['MessageId']}"
)
link_messages.append(df)
links_df = pd.concat(link_messages)
md(f"{len(links_df.index)} messages found:")
display(links_df.head())
Its likely at this stage in the investigation we still have large number of messages to investigate. For the next stage of our investigation we are going to use MSTICPy's IoC extraction capabilities to find the URLs in the messages and then look them up in our Threat Intelligence data to see if any are known to be suspicious.
# Lookup context to add to results
def perform_lookups(row):
channel_details = graph_provider.exec_query(
f"/teams/{row['channelIdentity.teamId']}/channels/{row['channelIdentity.channelId']}"
)
channel_name = channel_details["displayName"].iloc[0]
teams_details = graph_provider.exec_query(
f"/teams/{row['channelIdentity.teamId']}/"
)
teams_name = teams_details["displayName"].iloc[0]
user = graph_provider.exec_query(f"/users/{row['from.user.id']}/")
user_name = user["userPrincipalName"].iloc[0]
return pd.Series(
{
"UserPrincipalName": user_name,
"ChannelName": channel_name,
"TeamName": teams_name,
}
)
ioc_matches = links_df.mp_ioc.extract(columns=["body.content"], ioc_types=["url"])
ioc_matches["SourceIndex"] = pd.to_numeric(ioc_matches["SourceIndex"])
ioc_matches[ioc_matches["IoCType"] == "url"]
merged_ioc_df = pd.merge(
left=links_df,
right=ioc_matches[ioc_matches["IoCType"] == "url"],
how="right",
left_index=True,
right_on="SourceIndex",
)
ti = TILookup()
ti_hits = ti.lookup_iocs(merged_ioc_df["Observable"], providers=["XForce", "OTX"])
obs = ti_hits[ti_hits["Severity"].isin(["high", "warning"])]["Ioc"].unique()
merged_ioc_df["risky"] = np.where(merged_ioc_df["Observable"].isin(obs), True, False)
merged_ioc_df["SentByUserId"] = merged_ioc_df["from.user.id"]
merged_ioc_df["SentByUserName"] = merged_ioc_df["from.user.displayName"]
merged_ioc_df[["SentBy", "PostedToChannel", "PostedToTeam"]] = merged_ioc_df.apply(
perform_lookups, axis=1
)
md(f"{len(merged_ioc_df.index)} messages with URLs present in Threat Intelligence:")
display(
merged_ioc_df[
[
"createdDateTime",
"lastModifiedDateTime",
"SentBy",
"body.content",
"Observable",
"risky",
"PostedToTeam",
"PostedToChannel",
]
]
)
In the cells above we loook at links shared via Teams channels, however messages can also be shared in individual chats as well.
Looking at every chat message is infeasible but we can focus on chat messages sent by users who have shared malicious links in channels.
To do this we must first identify all of the chats those users are in, then get all the messages in those chats, and then look at any of those messages that have potential malicious links in them.
# Lookup context to add to results for chats
def perform_chat_lookups(user):
user = graph_provider.exec_query(f"/users/{user}/")
user_name = user["userPrincipalName"].iloc[0]
return user_name
# For each user in the above results, get thier chat messages
sending_users = merged_ioc_df["SentBy"].unique()
all_messages = []
for user in sending_users:
chats_df = graph_provider.exec_query(f"/users/{user}/chats")
chats = [message["id"] for message in chats_df["value"].iloc[0]]
for chat in chats:
messages = graph_provider.exec_query(f"/users/{user}/chats/{chat}/messages")
all_messages.append(messages)
chat_messages_df = pd.concat(all_messages)
# We can now parse out the message details and extract any URLs in them.
chat_messages_df = (
chat_messages_df["value"]
.apply(pd.Series)
.merge(chat_messages_df, right_index=True, left_index=True)
.drop(["value"], axis=1)
.melt(id_vars=["@odata.context", "@odata.nextLink"], value_name="value")
.drop("variable", axis=1)
.dropna()
)
chat_messages_df = pd.json_normalize(chat_messages_df["value"])
chat_messages_df.dropna(subset=["body.content"], inplace=True)
chat_messages_df["content"] = chat_messages_df["body.content"].astype(str)
chat_ioc_matches = chat_messages_df.mp_ioc.extract(
columns=["content"], ioc_types=["url"]
)
chat_ioc_matches["SourceIndex"] = pd.to_numeric(chat_ioc_matches["SourceIndex"])
chat_ioc_matches[chat_ioc_matches["IoCType"] == "url"]
merged_chat_ioc_df = pd.merge(
left=chat_messages_df,
right=chat_ioc_matches[chat_ioc_matches["IoCType"] == "url"],
how="right",
left_index=True,
right_on="SourceIndex",
)
# We can now look up those URLs in TI and filter on these items
chat_ti_hits = ti.lookup_iocs(
merged_chat_ioc_df["Observable"], providers=["XForce", "OTX"]
)
chat_obs = chat_ti_hits[chat_ti_hits["Severity"].isin(["high", "warning"])][
"Ioc"
].unique()
merged_chat_ioc_df["risky"] = np.where(
merged_chat_ioc_df["Observable"].isin(chat_obs), True, False
)
merged_chat_ioc_df[merged_chat_ioc_df["risky"] == True]
merged_chat_ioc_df["SentByUserId"] = merged_chat_ioc_df["from.user.id"]
merged_chat_ioc_df["SentByUserName"] = merged_chat_ioc_df["from.user.displayName"]
merged_chat_ioc_df["PostedToChannel"] = merged_chat_ioc_df["chatId"]
merged_chat_ioc_df["PostedToTeam"] = "NaN"
merged_chat_ioc_df["SentBy"] = merged_chat_ioc_df["from.user.id"].apply(
perform_chat_lookups
)
all_iocs_df = pd.concat([merged_chat_ioc_df, merged_ioc_df])
md(
f"{len(merged_chat_ioc_df.index)} chat messages with URLs present in Threat Intelligence:"
)
display(
merged_chat_ioc_df[
[
"createdDateTime",
"lastModifiedDateTime",
"SentBy",
"body.content",
"Observable",
"risky",
"PostedToChannel",
]
]
)
We can now summarize our results to see who has been posting malicious URLs and where to.
md(
f"{len(all_iocs_df['Observable'].unique())} malicious URL(s) sent by {len(all_iocs_df['SentBy'].unique())} user(s) in {len(all_iocs_df['id'].unique())} message(s), across {len(all_iocs_df['PostedToChannel'].astype(str).unique())} channel(s)"
)
display("Malicious URLs sent:")
display(all_iocs_df.groupby(["SentBy", "Observable"]).agg({"PostedToChannel": list}))
risky_users = all_iocs_df[all_iocs_df["risky"] == True]["SentBy"].unique()
risky_messages = all_iocs_df[all_iocs_df["SentBy"].isin(risky_users)]
md("Risky Messages:", "bold")
display(risky_messages["body.content"])
md("Channels with risky messages:", "bold")
display(risky_messages["channelIdentity.channelId"].unique())
# Create a graph of users, urls, messages, channels to show the connections between them
%matplotlib inline
import matplotlib.pyplot as plt
import networkx as nx
all_iocs_df.mp_plot.timeline(
source_columns=["Observable", "SentBy", "PostedToChannel"],
time_column="createdDateTime",
group_by="risky",
title="Timeline of message posts",
)
md("Graph of events:", "bold")
G = nx.from_pandas_edgelist(
all_iocs_df, "SentBy", "Observable", edge_attr=["PostedToChannel", "body.content"]
)
for row in all_iocs_df[["Observable", "PostedToChannel", "body.content"]].iterrows():
G.add_edge(row[1]["Observable"], str(row[1]["PostedToChannel"]))
fig = plt.figure(1, figsize=(20, 20))
nx.draw(G, with_labels=True, font_size=12)
We can now expand our search to look for other hosts that may have visited these URLs to further expand the investigation scope. By using CommonSecurityLogs from Sentinel with MDE's DeviceNetworkInformation we can identify the hosts making these connections.
# Hosts that have visited these URLs
dns_matches = ioc_extractor.extract_df(
all_iocs_df, columns=["Observable"], ioc_types=["dns"]
)
dns_matches["Observable"].unique()
query = f"""
let urls = dynamic({all_iocs_df["Observable"].unique().tolist()});
CommonSecurityLog
| where TimeGenerated > ago(7d)
| where RequestURL in (urls)
| extend timekey = bin(TimeGenerated, 1h)
| join kind=inner (DeviceNetworkInfo
| where TimeGenerated > ago(7d)
| mv-expand IPAddresses
| extend device_ip = tostring(IPAddresses.IPAddress)
| extend timekey = bin(TimeGenerated, 1h)) on $left.SourceIP == $right.device_ip, timekey
| project-reorder DeviceName1, timekey
| summarize max(timekey) by DeviceName1"""
connection_events = sent_provider.exec_query(query)
display(connection_events)
# See if URLS were seen elsewhere i.e. Office Events, Alerts and create timeline
# Alerts with any of the URLs - timeline
alerts_query = f"""let urls = dynamic([{all_iocs_df["Observable"].unique().tolist()}]);
SecurityAlert
| mv-expand todynamic(Entities)
| where tostring(Entities.Type) =~ "url"
| evaluate bag_unpack(Entities, "Entities_")
| where Entities_Url in (urls)"""
alert_events = sent_provider.exec_query(alerts_query)
display(alert_events)
if not alert_events.empty:
alert_events.mp_plot.timeline(
source_columns=["Entities_Url"],
time_column="TimeGenerated",
group_by="Entities_Url",
title="Timeline of alerts",
)
The following are a set of entities worthy of further investigation.
md("Users who sent suspicious URLs:", "bold")
display(list(risky_users))
md("Hosts that accessed suspcious Urls:", "bold")
display(
list(ss_df["DeviceName"].unique()) + list(connection_events["DeviceName1"].unique())
)
md("Suspicious URLs that were shared:", "bold")
display(list(all_iocs_df["Observable"].unique()))