Инициализация и получение данных¶

Ноутбук демонстрирует пример запроса и работы с данными MyTracker.

Внимание! Чтобы всё заработало, в ячейку ниже вам необходио вписать UserId и SecretKey! Получить их можно в личном кабинете, если у вас есть права на экспорт сырых данных. Пожалуйста, проконсультируйтесь с документацией, как сделать это.

Таймстампы все в UTC исходно, мы их приводим в Europe/Moscow и специально даём суффикс полям _MSK!

In [27]:

API_USER_ID = 49759
API_SECRET_KEY = 'qwert'

In [2]:

%pip install mytracker_export_api

Requirement already satisfied: mytracker_export_api in /opt/conda/lib/python3.11/site-packages (1.0.5)
Requirement already satisfied: requests>=2.20.0 in /opt/conda/lib/python3.11/site-packages (from mytracker_export_api) (2.31.0)
Requirement already satisfied: pandas>=1.1.0 in /opt/conda/lib/python3.11/site-packages (from mytracker_export_api) (2.0.3)
Requirement already satisfied: urllib3<2,>=1.26.0 in /opt/conda/lib/python3.11/site-packages (from mytracker_export_api) (1.26.19)
Requirement already satisfied: python-dateutil>=2.8.2 in /opt/conda/lib/python3.11/site-packages (from pandas>=1.1.0->mytracker_export_api) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in /opt/conda/lib/python3.11/site-packages (from pandas>=1.1.0->mytracker_export_api) (2023.3)
Requirement already satisfied: tzdata>=2022.1 in /opt/conda/lib/python3.11/site-packages (from pandas>=1.1.0->mytracker_export_api) (2023.3)
Requirement already satisfied: numpy>=1.21.0 in /opt/conda/lib/python3.11/site-packages (from pandas>=1.1.0->mytracker_export_api) (1.24.4)
Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.11/site-packages (from requests>=2.20.0->mytracker_export_api) (3.2.0)
Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.11/site-packages (from requests>=2.20.0->mytracker_export_api) (3.4)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.11/site-packages (from requests>=2.20.0->mytracker_export_api) (2023.5.7)
Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.11/site-packages (from python-dateutil>=2.8.2->pandas>=1.1.0->mytracker_export_api) (1.16.0)
WARNING: There was an error checking the latest version of pip.
Note: you may need to restart the kernel to use updated packages.

In [3]:

from mytracker_export_api import MyTracker
params = {
    'dateFrom': '2024-06-24',
    'dateTo': '2024-06-25',
    'selectors': 'eventName,idAppVersionTitle,idOsTitle,tsEvent,tsInsert,idDevice,params.name,params.value,customUserId',
    'idApp': 61475,
#    'idApp': '61389,61390,61475,61476,80840,80842,80843,95584',
    'event': 'customEvents'
}
client = MyTracker(api_user_id=API_USER_ID, api_secret_key=API_SECRET_KEY)
df = client.get_raw_data(params)

In [4]:

df.head()

Out[4]:

	eventName	idAppVersionTitle	idOsTitle	tsEvent	tsInsert	idDevice	params.name	params.value	customUserId
0	mt_login	2.19.2-d1a60611	Android	1719296521	1719301344	52344631831	[]	[]	[f2fa6186-1c44-4f82-b62b-34ac023d17b1,a5086f93...
1	mt_login	2.19.2-d1a60611	Android	1719296530	1719301344	52344631831	[]	[]	[f2fa6186-1c44-4f82-b62b-34ac023d17b1,a5086f93...
2	mt_login	2.19.1-08f155b8	Android	1719296778	1719301344	52344631831	[]	[]	[432b4ca1-82f3-4cf7-a714-f08adfdb4343,27d6f62a...
3	mt_login	2.19.1-08f155b8	Android	1719296906	1719301344	52344631831	[]	[]	[432b4ca1-82f3-4cf7-a714-f08adfdb4343,27d6f62a...
4	mt_login	2.20.0-02c55807	Android	1719306146	1719312270	86868404654	[]	[]	[04203afa-27b3-4844-95da-8238bf69df4a,2fed9ec3...

In [5]:

display(df.head())

	eventName	idAppVersionTitle	idOsTitle	tsEvent	tsInsert	idDevice	params.name	params.value	customUserId
0	mt_login	2.19.2-d1a60611	Android	1719296521	1719301344	52344631831	[]	[]	[f2fa6186-1c44-4f82-b62b-34ac023d17b1,a5086f93...
1	mt_login	2.19.2-d1a60611	Android	1719296530	1719301344	52344631831	[]	[]	[f2fa6186-1c44-4f82-b62b-34ac023d17b1,a5086f93...
2	mt_login	2.19.1-08f155b8	Android	1719296778	1719301344	52344631831	[]	[]	[432b4ca1-82f3-4cf7-a714-f08adfdb4343,27d6f62a...
3	mt_login	2.19.1-08f155b8	Android	1719296906	1719301344	52344631831	[]	[]	[432b4ca1-82f3-4cf7-a714-f08adfdb4343,27d6f62a...
4	mt_login	2.20.0-02c55807	Android	1719306146	1719312270	86868404654	[]	[]	[04203afa-27b3-4844-95da-8238bf69df4a,2fed9ec3...

Предобработка¶

Конвертация дат, фильтрация

Convert params to dict¶

In [6]:

import ast
df['params'] = df.apply(lambda it: dict(zip(ast.literal_eval(it['params.name']), ast.literal_eval(it['params.value']))), axis=1)
df.drop(columns=['params.name', 'params.value'], axis=1, inplace=True)

Extract account_id¶

In [7]:

def exctr_account_id(s: str):
    parts = re.split('[\[\,]+', str(s))
    try:
        return parts[1]
    except IndexError:
        return ''

import re
#df['account_id'] = df['customUserId'].apply(lambda it: re.split('[\[\,]+', it))
df['account_id'] = df['customUserId'].apply(exctr_account_id)
df.drop(columns=['customUserId'], axis=1, inplace=True)
df

Out[7]:

	eventName	idAppVersionTitle	idOsTitle	tsEvent	tsInsert	idDevice	params	account_id
0	mt_login	2.19.2-d1a60611	Android	1719296521	1719301344	52344631831	{}	f2fa6186-1c44-4f82-b62b-34ac023d17b1
1	mt_login	2.19.2-d1a60611	Android	1719296530	1719301344	52344631831	{}	f2fa6186-1c44-4f82-b62b-34ac023d17b1
2	mt_login	2.19.1-08f155b8	Android	1719296778	1719301344	52344631831	{}	432b4ca1-82f3-4cf7-a714-f08adfdb4343
3	mt_login	2.19.1-08f155b8	Android	1719296906	1719301344	52344631831	{}	432b4ca1-82f3-4cf7-a714-f08adfdb4343
4	mt_login	2.20.0-02c55807	Android	1719306146	1719312270	86868404654	{}	04203afa-27b3-4844-95da-8238bf69df4a
...	...	...	...	...	...	...	...	...
7664	swipe	2.20.0-ddc6a301	Android	1719309083	1719317825	70284455492	{'code': 'LOYALTY_BALANCE.ACCOUNTS_WIDGETS', '...	43807c7d-14c9-41d0-9f0d-3041524180e6
7665	swipe	2.20.0-ddc6a301	Android	1719309089	1719317825	70284455492	{'code': 'LOYALTY_BALANCE.TRANSACTION_FILTERS'...	43807c7d-14c9-41d0-9f0d-3041524180e6
7666	swipe	2.20.0-ddc6a301	Android	1719314829	1719318680	70284455492	{'code': 'PROFILE.PROFILE', 'name': 'Profile',...	43807c7d-14c9-41d0-9f0d-3041524180e6
7667	swipe	2.20.0-ddc6a301	Android	1719314830	1719318680	70284455492	{'code': 'PROFILE.PROFILE', 'name': 'Profile',...	43807c7d-14c9-41d0-9f0d-3041524180e6
7668	swipe	2.20.0-ddc6a301	Android	1719314831	1719318680	70284455492	{'code': 'PROFILE.PROFILE', 'name': 'Profile',...	43807c7d-14c9-41d0-9f0d-3041524180e6

7669 rows × 8 columns

Преобразование timestampt в дату и время¶

In [8]:

import pandas as pd
# tz_localize/tz_convert by https://stackoverflow.com/questions/42826388/using-time-zone-in-pandas-to-datetime/42826430#42826430
#df['eventDT'] = pd.to_datetime(df['tsEvent'], unit='s').dt.tz_localize('UTC')
df['eventDT_MSK'] = pd.to_datetime(df['tsEvent'], unit='s').dt.tz_localize('UTC').dt.tz_convert('Europe/Moscow')
df['eventInsertDT_MSK'] = pd.to_datetime(df['tsInsert'], unit='s').dt.tz_localize('UTC').dt.tz_convert('Europe/Moscow')

Считаем время "долёта" событий¶

In [9]:

df['ts_diff'] = df['eventInsertDT_MSK'] - df['eventDT_MSK']

Сортируем по времени событий¶

In [10]:

df.sort_values(by=['tsEvent'], inplace=True)

Фильтрация и примеры выборки¶

Фильтрация по имени события eventName¶

In [11]:

df['_MyTrackerIdTestEventDEBUG' == df['eventName']]

Out[11]:

	eventName	idAppVersionTitle	idOsTitle	tsEvent	tsInsert	idDevice	params	account_id	eventDT_MSK	eventInsertDT_MSK	ts_diff

Фильтрация по account_id¶

In [14]:

df['262c20d2-2d71-40bf-aa86-908f27e08c48' == df['account_id']]

Out[14]:

	eventName	idAppVersionTitle	idOsTitle	tsEvent	tsInsert	idDevice	params	account_id	eventDT_MSK	eventInsertDT_MSK	ts_diff

Фильтрация по номеру девайса¶

In [24]:

df[1234 == df['idDevice']]

Out[24]:

	eventName	idAppVersionTitle	idOsTitle	tsEvent	tsInsert	idDevice	params	account_id	eventDT_MSK	eventInsertDT_MSK	ts_diff

Фильтрация по двум параметрам - девайсу И событию вместе¶

In [ ]:

df[(df['eventName'] == 'press') & (1234 == df['idDevice'])]

Фильтрация по ИЛИ - несколько событий¶

In [12]:

df[ ('_MyTrackerIdTestEventDEBUG' == df['eventName']) | ('_MyTrackerIdTestEventDEBUG_BeforeInitMyTracker' == df['eventName']) ]

Out[12]:

	eventName	idAppVersionTitle	idOsTitle	tsEvent	tsInsert	idDevice	params	account_id	eventDT_MSK	eventInsertDT_MSK	ts_diff

Примеры вывода и форматирования¶

Вывод определённых (выбранных) столбцов¶

In [20]:

# Александр Петров дев
df[['eventName', 'account_id', 'ts_diff']]

Out[20]:

	eventName	account_id	ts_diff
4713	close	b59243b4-9581-4f09-9844-1e6572d8622d	0 days 04:51:02
4991	press	b59243b4-9581-4f09-9844-1e6572d8622d	0 days 04:50:56
4468	channels_channel_open	b59243b4-9581-4f09-9844-1e6572d8622d	0 days 04:50:56
4842	main_open	b59243b4-9581-4f09-9844-1e6572d8622d	0 days 00:41:15
4843	main_open	b59243b4-9581-4f09-9844-1e6572d8622d	0 days 00:40:45
...	...	...	...
5739	swipe	c356eb4c-5a33-48e2-b447-963ff0064328	0 days 00:46:48
5740	swipe	c356eb4c-5a33-48e2-b447-963ff0064328	0 days 00:46:47
5722	main_open	c356eb4c-5a33-48e2-b447-963ff0064328	0 days 00:46:41
5726	open	c356eb4c-5a33-48e2-b447-963ff0064328	0 days 00:46:41
5730	radio_widget	c356eb4c-5a33-48e2-b447-963ff0064328	0 days 00:46:41

7669 rows × 3 columns

Вывод всех, кроме выбранных столбцов¶

In [25]:

# By https://stackoverflow.com/questions/29763620/how-to-select-all-columns-except-one-in-pandas/53075169#53075169
df.loc[:, ~df.columns.isin(['params', 'account_id'])]

Out[25]:

	eventName	idAppVersionTitle	idOsTitle	tsEvent	tsInsert	idDevice	eventDT_MSK	eventInsertDT_MSK	ts_diff
4713	close	3.0.0	Android	1719196195	1719213657	46509549178	2024-06-24 05:29:55+03:00	2024-06-24 10:20:57+03:00	0 days 04:51:02
4991	press	3.0.0	Android	1719196201	1719213657	46509549178	2024-06-24 05:30:01+03:00	2024-06-24 10:20:57+03:00	0 days 04:50:56
4468	channels_channel_open	3.0.0	Android	1719196201	1719213657	46509549178	2024-06-24 05:30:01+03:00	2024-06-24 10:20:57+03:00	0 days 04:50:56
4842	main_open	3.0.0	Android	1719211182	1719213657	46509549178	2024-06-24 09:39:42+03:00	2024-06-24 10:20:57+03:00	0 days 00:41:15
4843	main_open	3.0.0	Android	1719211212	1719213657	46509549178	2024-06-24 09:40:12+03:00	2024-06-24 10:20:57+03:00	0 days 00:40:45
...	...	...	...	...	...	...	...	...	...
5739	swipe	2.20.0-61b7e141	Android	1719327881	1719330689	86898910861	2024-06-25 18:04:41+03:00	2024-06-25 18:51:29+03:00	0 days 00:46:48
5740	swipe	2.20.0-61b7e141	Android	1719327882	1719330689	86898910861	2024-06-25 18:04:42+03:00	2024-06-25 18:51:29+03:00	0 days 00:46:47
5722	main_open	2.20.0-61b7e141	Android	1719327888	1719330689	86898910861	2024-06-25 18:04:48+03:00	2024-06-25 18:51:29+03:00	0 days 00:46:41
5726	open	2.20.0-61b7e141	Android	1719327888	1719330689	86898910861	2024-06-25 18:04:48+03:00	2024-06-25 18:51:29+03:00	0 days 00:46:41
5730	radio_widget	2.20.0-61b7e141	Android	1719327888	1719330689	86898910861	2024-06-25 18:04:48+03:00	2024-06-25 18:51:29+03:00	0 days 00:46:41

7669 rows × 9 columns