Here are some tips and tricks for ease of use and productive experience with Aerospike notebooks.
The Jupyter Notebook provides "a web-based application suitable for capturing the whole computation process: developing, documenting, and executing code, as well as communicating the results". New to notebooks? Here is one source to learn more about the Jupyter Notebook.
Visit Aerospike notebooks repo to find additional Aerospike notebooks. To run another notebook, download the notebook from the repo to your local machine, and then click on File->Open, and select Upload.
Use the "!" line magic and "%%bash" cell magic to access shell commands. That is, you can access a shell command on any line by prefixing it with a "!", and an entire cell can have bash shell commands if it starts with "%%bash". Here are some examples:
# Accessing shell commands
!ps
!whoami
# Start the Aerospike database.
!asd >& /dev/null
PID TTY TIME CMD 10456 pts/3 00:00:00 ps jovyan
%%bash
# bash cell
# Check if the Aerospike database is running.
pgrep -x asd >/dev/null && echo "Aerospike database is running" || echo "**Aerospike database is not running!**"
ps -axu | grep asd
Aerospike database is running jovyan 25 1.8 8.5 4090004 175020 ? Ssl Feb28 238:27 /usr/bin/asd --config-file /etc/aerospike/aerospike.conf jovyan 10468 0.0 0.0 1028 248 ? R 23:43 0:00 grep asd
The Java kernel currently does not have a robust command shell access. The line magic %sh is limited in what it supports. For example, tools like aql or asadm that take arguments do not work, for example: ‘aql -c “select * from test.demo”’. Actions such as changing the server config or tailing server log may not be possible through the Java kernel’s %sh escape.
There are multiple ways to access a functional shell from a Java notebook:
Use a terminal window. From the directory view in the notebook (if in a notebook view, launch File->Open), there is a New menu on the top right side. Select New->Terminal for a fully functional shell.
Use Python kernel’s shell magic. Start a new Python notebook (File->New Notebook->Python 3) and use the “!!” cell magic or “!” line magic to access shell commands. The Python kernel’s shell access works better than Java kernel’s at this point.
It is useful to examine the server log. Assuming it is located at /var/log/aerospike/aerospike.log, and you have the permissions, you can run the following to view the last 10 lines of the log. (Adjust the log path to your setting.)
# View the last 10 lines of the log:
!echo "End of server log:"; tail -10 /var/log/aerospike/aerospike.log
End of server log: tail: cannot open '/var/log/aerospike/aerospike.log' for reading: No such file or directory
# Insert a record in set "demo" in namsepace "test" with Primary Key (PK) 1 and a bin or field "testbin"
# with value "hello world!".
!aql -c "INSERT INTO test.demo (PK, 'testbin') VALUES (1, 'hello world!')"
# View all records in the set.
!aql -c "SELECT * FROM test.demo"
# Delete the record
!aql -c "DELETE FROM test.demo WHERE PK = 1"
!aql -c "SELECT * FROM test.demo"
INSERT INTO test.demo (PK, 'testbin') VALUES (1, 'hello world!') OK, 1 record affected. SELECT * FROM test.demo +----------------+ | testbin | +----------------+ | "hello world!" | +----------------+ 1 row in set (0.167 secs) OK DELETE FROM test.demo WHERE PK = 1 OK, 1 record affected. SELECT * FROM test.demo 0 rows in set (0.151 secs) OK
# Show the features enabled in this database.
!asadm -e "features"
# Display summary info for the cluster
!asadm -e "summary"
# View the config
!asadm -e "show config"
Seed: [('127.0.0.1', 3000, None)] Config_file: /home/jovyan/.aerospike/astools.conf, /etc/aerospike/astools.conf ~Features (2021-03-09 23:43:06 UTC)~ NODE : 1fd218aa4d65:3000 AGGREGATION : YES BATCH : YES INDEX-ON-DEVICE: NO INDEX-ON-PMEM : NO KVS : YES LDT : NO QUERY : YES RACK-AWARE : NO SC : NO SCAN : YES SECURITY : NO SINDEX : NO TLS (FABRIC) : NO TLS (HEARTBEAT): NO TLS (SERVICE) : NO UDF : YES XDR DESTINATION: NO XDR SOURCE : NO Seed: [('127.0.0.1', 3000, None)] Config_file: /home/jovyan/.aerospike/astools.conf, /etc/aerospike/astools.conf Cluster ======= 1. Server Version : E-5.4.0.1 2. OS Version : Ubuntu 20.04.1 LTS (4.19.121-linuxkit) 3. Cluster Size : 1 4. Devices : Total 1, per-node 1 5. Memory : Total 1.000 GB, 0.00% used (0.000 B), 100.00% available (1.000 GB) 6. Disk : Total 4.000 GB, 0.00% used (0.000 B), 99.00% available contiguous space (3.960 GB) 7. Usage (Unique Data): None 8. Active Namespaces : 0 of 1 9. Features : Aggregation, Batch, KVS, Query, Scan, UDF ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~Namespaces~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Namespace Devices Memory Disk Replication Rack Master . (Total,Per-Node) (Total,Used%,Avail%) (Total,Used%,Avail%) Factor Aware Objects test (1, 1) (1.000 GB, 0.00, 100.00) (4.000 GB, 0.00, 99.00) 2 False 0.000 Number of rows: 1 Seed: [('127.0.0.1', 3000, None)] Config_file: /home/jovyan/.aerospike/astools.conf, /etc/aerospike/astools.conf ~~~~~~~~~Service Configuration (2021-03-09 23:43:08 UTC)~~~~~~~~~ NODE : 1fd218aa4d65:3000 advertise-ipv6 : false auto-pin : none batch-index-threads : 4 batch-max-buffers-per-queue : 255 batch-max-requests : 5000 batch-max-unused-buffers : 256 batch-without-digests : false cluster-name : null debug-allocations : none disable-udf-execution : false enable-benchmarks-fabric : false enable-health-check : false enable-hist-info : false enable-ldap : false enable-security : false fabric.address : any fabric.channel-bulk-fds : 2 fabric.channel-bulk-recv-threads : 4 fabric.channel-ctrl-fds : 1 fabric.channel-ctrl-recv-threads : 4 fabric.channel-meta-fds : 1 fabric.channel-meta-recv-threads : 4 fabric.channel-rw-fds : 8 fabric.channel-rw-recv-pools : 1 fabric.channel-rw-recv-threads : 16 fabric.keepalive-enabled : true fabric.keepalive-intvl : 1 fabric.keepalive-probes : 10 fabric.keepalive-time : 1 fabric.latency-max-ms : 5 fabric.port : 3001 fabric.recv-rearm-threshold : 1024 fabric.send-threads : 8 fabric.tls-name : null fabric.tls-port : 0 feature-key-file : /etc/aerospike/features.conf heartbeat.address : any heartbeat.connect-timeout-ms : 500 heartbeat.interval : 150 heartbeat.mode : mesh heartbeat.mtu : 1500 heartbeat.port : 3002 heartbeat.protocol : v3 heartbeat.timeout : 10 heartbeat.tls-name : null heartbeat.tls-port : 0 indent-allocations : false info-threads : 16 info.address : any info.port : 3003 keep-caps-ssd-health : false ldap-login-threads : 8 ldap.disable-tls : false ldap.polling-period : 300 ldap.query-base-dn : null ldap.query-user-dn : null ldap.query-user-password-file : null ldap.role-query-base-dn : null ldap.role-query-search-ou : false ldap.server : null ldap.session-ttl : 86400 ldap.tls-ca-file : null ldap.token-hash-method : sha-256 ldap.user-dn-pattern : null ldap.user-query-pattern : null log-local-time : false log-millis : false microsecond-histograms : false migrate-fill-delay : 0 migrate-max-num-incoming : 4 migrate-threads : 1 min-cluster-size : 1 node-id : BB9020011AC4202 node-id-interface : null paxos-single-replica-limit : 1 pidfile : /var/run/aerospike/asd.pid privilege-refresh-period : 300 proto-fd-idle-ms : 0 proto-fd-max : 15000 proto-slow-netio-sleep-ms : 1 query-batch-size : 100 query-buf-size : 2097152 query-bufpool-size : 256 query-in-transaction-thread : false query-long-q-max-size : 500 query-microbenchmark : false query-pre-reserve-partitions : false query-priority : 10 query-priority-sleep-us : 1 query-rec-count-bound : 18446744073709551615 query-req-in-query-thread : false query-req-max-inflight : 100 query-short-q-max-size : 500 query-threads : 6 query-threshold : 10 query-untracked-time-ms : 1000 query-worker-threads : 15 report-authentication-sinks : 0 report-data-op-sinks : 0 report-sys-admin-sinks : 0 report-user-admin-sinks : 0 report-violation-sinks : 0 run-as-daemon : true scan-max-done : 100 scan-threads-limit : 128 service-threads : 4 service.access-port : 0 service.address : any service.alternate-access-port : 0 service.port : 3000 service.tls-access-port : 0 service.tls-alternate-access-port: 0 service.tls-name : null service.tls-port : 0 sindex-builder-threads : 4 sindex-gc-max-rate : 50000 sindex-gc-period : 10 stay-quiesced : false syslog-local : -1 ticker-interval : 10 transaction-max-ms : 1000 transaction-retry-ms : 1002 vault-ca : null vault-path : null vault-token-file : null vault-url : null work-directory : /opt/aerospike ~~~Network Configuration (2021-03-09 23:43:08 UTC)~~~~ NODE : 1fd218aa4d65:3000 fabric.address : any fabric.channel-bulk-fds : 2 fabric.channel-bulk-recv-threads : 4 fabric.channel-ctrl-fds : 1 fabric.channel-ctrl-recv-threads : 4 fabric.channel-meta-fds : 1 fabric.channel-meta-recv-threads : 4 fabric.channel-rw-fds : 8 fabric.channel-rw-recv-pools : 1 fabric.channel-rw-recv-threads : 16 fabric.keepalive-enabled : true fabric.keepalive-intvl : 1 fabric.keepalive-probes : 10 fabric.keepalive-time : 1 fabric.latency-max-ms : 5 fabric.port : 3001 fabric.recv-rearm-threshold : 1024 fabric.send-threads : 8 fabric.tls-name : null fabric.tls-port : 0 heartbeat.address : any heartbeat.connect-timeout-ms : 500 heartbeat.interval : 150 heartbeat.mode : mesh heartbeat.mtu : 1500 heartbeat.port : 3002 heartbeat.protocol : v3 heartbeat.timeout : 10 heartbeat.tls-name : null heartbeat.tls-port : 0 info.address : any info.port : 3003 service.access-port : 0 service.address : any service.alternate-access-port : 0 service.port : 3000 service.tls-access-port : 0 service.tls-alternate-access-port: 0 service.tls-name : null service.tls-port : 0 ~~~~~~~~~~test Namespace Configuration (2021-03-09 23:43:08 UTC)~~~~~~~~~ NODE : 1fd218aa4d65:3000 allow-ttl-without-nsup : false background-scan-max-rps : 10000 conflict-resolution-policy : generation conflict-resolve-writes : false data-in-index : false default-ttl : 2592000 disable-cold-start-eviction : false disable-write-dup-res : false disallow-null-setname : false enable-benchmarks-batch-sub : false enable-benchmarks-ops-sub : false enable-benchmarks-read : false enable-benchmarks-udf : false enable-benchmarks-udf-sub : false enable-benchmarks-write : false enable-hist-proxy : false evict-hist-buckets : 10000 evict-tenths-pct : 5 geo2dsphere-within.earth-radius-meters : 6371000 geo2dsphere-within.level-mod : 1 geo2dsphere-within.max-cells : 12 geo2dsphere-within.max-level : 30 geo2dsphere-within.min-level : 1 geo2dsphere-within.strict : true high-water-disk-pct : 0 high-water-memory-pct : 0 ignore-migrate-fill-delay : false index-stage-size : 1073741824 index-type : shmem memory-size : 1073741824 migrate-order : 5 migrate-retransmit-ms : 5000 migrate-sleep : 1 nsid : 0 nsup-hist-period : 3600 nsup-period : 120 nsup-threads : 1 partition-tree-sprigs : 256 prefer-uniform-balance : true rack-id : 0 read-consistency-level-override : off reject-non-xdr-writes : false reject-xdr-writes : false replication-factor : 2 sindex.num-partitions : 32 single-bin : false single-scan-threads : 4 stop-writes-pct : 90 storage-engine : device storage-engine.cache-replica-writes : false storage-engine.cold-start-empty : false storage-engine.commit-min-size : 0 storage-engine.commit-to-device : false storage-engine.compression : none storage-engine.compression-level : 0 storage-engine.data-in-memory : true storage-engine.defrag-lwm-pct : 50 storage-engine.defrag-queue-min : 0 storage-engine.defrag-sleep : 1000 storage-engine.defrag-startup-minimum : 10 storage-engine.direct-files : false storage-engine.disable-odsync : false storage-engine.enable-benchmarks-storage : false storage-engine.encryption-key-file : null storage-engine.file[0] : /opt/aerospike/data/test.dat storage-engine.filesize : 4294967296 storage-engine.flush-max-ms : 1000 storage-engine.max-write-cache : 67108864 storage-engine.min-avail-pct : 5 storage-engine.post-write-queue : 0 storage-engine.read-page-cache : false storage-engine.scheduler-mode : null storage-engine.serialize-tomb-raider : false storage-engine.sindex-startup-device-scan: false storage-engine.tomb-raider-sleep : 1000 storage-engine.write-block-size : 1048576 strong-consistency : false strong-consistency-allow-expunge : false tomb-raider-eligible-age : 86400 tomb-raider-period : 86400 transaction-pending-limit : 20 truncate-threads : 4 write-commit-level-override : off xdr-bin-tombstone-ttl : 86400 xdr-tomb-raider-period : 120 xdr-tomb-raider-threads : 1
Visit Aerospike notebooks repo to run additional Aerospike notebooks. To run a different notebook, download the notebook from the repo to your local machine, and then click on File->Open, and select Upload.