Notebook

Installation¶

Note: If you are on Binder, you don't need to execute the following command.

In [ ]:

!pip install nexusforge[linking_sklearn]

In [ ]:

import os 

def full_path_relative_to_root(path): 
    return os.path.join(os.path.abspath('../../..'), path)

Configuration¶

This notebook presents a set of configuation options to set up when creating a knowledge graph forge session. Refer to the Nexus Forge docs to learn more about all the possible configuration options.

In [ ]:

config = dict()

Configure for Demo¶

This configuration is for testing Nexus Forge features without using or deploying a persistent store. Not all features are accessible with the demo configuration. The demo configuration is therefore not recommendeded for production use.

Model¶

In [ ]:

config['Model'] = {
    "name": "DemoModel",
    "origin": "directory",
    "source": full_path_relative_to_root("tests/data/demo-model/"),
}

Store¶

In [ ]:

config["Store"] = {
    "name": "DemoStore",
    "model": {"name": "DemoModel"},
    "versioned_id_template": "{x.id}?_version={x._store_metadata.version}"
}

Resolvers¶

sourced from a directory¶

In [ ]:

config["Resolvers"] = {
    "terms": [
        {
            "resolver": "DemoResolver",
            "origin": "directory",
            "source": full_path_relative_to_root("tests/data/demo-resolver/"),
            "targets": [
                {
                    "identifier": "sexontology",
                    "bucket": "sex.json"
                }
            ],
            "result_resource_mapping": full_path_relative_to_root("examples/configurations/demo-resolver/term-to-resource-mapping.hjson")
        }
    ],
    "entities": [
        {
            "resolver": "DemoResolver",
            "origin": "directory",
            "source": full_path_relative_to_root("tests/data/demo-resolver/"),
            "targets": [
                {
                    "identifier": "agents",
                    "bucket": "agents.json"
                }
            ],
            "result_resource_mapping": full_path_relative_to_root("examples/configurations/demo-resolver/entity-to-resource-mapping.hjson")
        }
    ],
    "schemaorg": [
        {
            "resolver": "EntityLinkerSkLearn from kgentitylinkingsklearn",
            "origin": "directory",
            "source": full_path_relative_to_root("examples/data/"),
            "targets": [
                {
                    "identifier": "terms",
                    "bucket": "tfidfvectorizer_model_schemaorg_linking"
                }
            ],
            "result_resource_mapping": full_path_relative_to_root("examples/configurations/entitylinking-resolver/entitylinking-mapper.hjson")
        }
    ]
}

Configure Nexus Forge to use with Blue Brain Nexus Delta as a Store¶

Create a project to work with in the BlueBrainNexus sandbox¶

Get a token¶

The Nexus sandbox application can be used to login and get a token.

Step 1: From the opened web page, click on the login button on the right corner and follow the instructions.

Step 2: At the end you’ll see a token button on the right corner. Click on it to copy the token.

In [ ]:

import getpass

token = getpass.getpass(prompt="Token")

Set a project name¶

In [ ]:

endpoint = "https://sandbox.bluebrainnexus.io/v1"
org ="github-users"

project = getpass.getpass(prompt="Github username") # Provide here the automatically created project name corresponding to your Github login when you logged in the Nexus sandbox instance.

Model¶

RDFModel¶

This model supports the W3C SHACL schema language. Let use examples of SHACL schemas from https://github.com/INCF/neuroshapes. SHACL schemas can be loaded either from a directory or from a store.

sourced from BlueBrainNexus store¶

In [ ]:

shacl_schema_bucket = "neurosciencegraph/datamodels"

In [ ]:

config['Model'] = {
    "name": "RdfModel",
    "origin": "store",
    "source": "BlueBrainNexus",
    "context": {
        "iri": "https://bbp.neuroshapes.org",
        "bucket": shacl_schema_bucket
    }
}

For the following tutorials please keep the following Model configuration:

sourced from a directory¶

In [ ]:

neuroshapes_path = full_path_relative_to_root("examples/models/neuroshapes")

In [ ]:

! rm -Rf $neuroshapes_path

In [ ]:

! git clone https://github.com/INCF/neuroshapes.git $neuroshapes_path

In [ ]:

! cp -R $neuroshapes_path/shapes/neurosciencegraph/datashapes/core/dataset $neuroshapes_path/shapes/neurosciencegraph/commons/
! cp -R $neuroshapes_path/shapes/neurosciencegraph/datashapes/core/activity $neuroshapes_path/shapes/neurosciencegraph/commons/
! cp -R $neuroshapes_path/shapes/neurosciencegraph/datashapes/core/entity $neuroshapes_path/shapes/neurosciencegraph/commons/
! cp -R $neuroshapes_path/shapes/neurosciencegraph/datashapes/core/ontology $neuroshapes_path/shapes/neurosciencegraph/commons/
! cp -R $neuroshapes_path/shapes/neurosciencegraph/datashapes/core/person $neuroshapes_path/shapes/neurosciencegraph/commons/
! cp -R $neuroshapes_path/shapes/neurosciencegraph/datashapes/core/contribution $neuroshapes_path/shapes/neurosciencegraph/commons/

In [ ]:

config['Model'] = {
    "name": "RdfModel",
    "origin": "directory",
    "source": f"{neuroshapes_path}/shapes/neurosciencegraph/commons/",
    "context": {
        "iri": full_path_relative_to_root("examples/models/neuroshapes_context.json")
    },
}

Store¶

In [ ]:

config["Store"] = {
    "name": "BlueBrainNexus",
    "endpoint": endpoint,
    "model": {"name": "RdfModel"}, 
    "searchendpoints":{
        "sparql":{
            "endpoint":"https://bluebrain.github.io/nexus/vocabulary/defaultSparqlIndex"
        },
        "elastic":{
            "endpoint":"https://bluebrain.github.io/nexus/vocabulary/defaultElasticSearchIndex"
        }
    },
    "bucket": f"{org}/{project}",
    "token": token,
    "vocabulary":{
        "metadata":{
            "iri": "https://bluebrain.github.io/nexus/contexts/metadata.json",
           "local_iri": "https://bluebrainnexus.io/contexts/metadata.json"
        }, 
        "namespace": "https://bluebrain.github.io/nexus/vocabulary/",
        "deprecated_property": "https://bluebrain.github.io/nexus/vocabulary/deprecated",
        "project_property": "https://bluebrain.github.io/nexus/vocabulary/project"
    },
    "max_connection": 50,
    "versioned_id_template": "{x.id}?rev={x._store_metadata._rev}",
    "file_resource_mapping": full_path_relative_to_root(
        "examples/configurations/nexus-store/file-to-resource-mapping.hjson"
    )
}

Resolvers¶

sourced from a store¶

In [ ]:

ontology_bucket = "neurosciencegraph/datamodels"

In [ ]:

config["Resolvers"] = {
    "terms": [
        {
            "resolver": "OntologyResolver",
            "origin": "store",
            "source": "BlueBrainNexus",
            "targets": [
                {
                    "identifier": "sexontology",
                    "bucket": ontology_bucket
                }
            ],
            "result_resource_mapping": full_path_relative_to_root(
                "examples/configurations/nexus-resolver/term-to-resource-mapping.hjson"
            )
        }
    ],
    "ontology": [
        {
            "resolver": "OntologyResolver",
            "origin": "store",
            "source": "BlueBrainNexus",
            "targets": [
                {
                    "identifier": "cells",
                    "bucket": ontology_bucket
                }
            ],
            "result_resource_mapping": full_path_relative_to_root(
                "examples/configurations/demo-resolver/term-to-resource-mapping.hjson"
            )
        }
    ],
    "entities": [
        {
            "resolver": "DemoResolver",
            "origin": "directory",
            "source": full_path_relative_to_root("tests/data/demo-resolver/"),
            "targets": [
                {
                    "identifier": "agents",
                    "bucket": "agents.json"
                }
            ],
            "result_resource_mapping": full_path_relative_to_root(
                "examples/configurations/demo-resolver/entity-to-resource-mapping.hjson"
            )
        }
    ],
    "schemaorg": [
        {
            "resolver": "EntityLinkerSkLearn from kgentitylinkingsklearn",
            "origin": "directory",
            "source": full_path_relative_to_root("examples/data/"),
            "targets": [
                {
                    "identifier": "terms",
                    "bucket": "tfidfvectorizer_model_schemaorg_linking"
                }
            ],
            "result_resource_mapping": full_path_relative_to_root(
                "examples/configurations/entitylinking-resolver/entitylinking-mapper.hjson"
            )
        }
    ]
}

sourced from a directory¶

In [ ]:

config["Resolvers"] = {
    "terms": [
        {
            "resolver": "DemoResolver",
            "origin": "directory",
            "source": full_path_relative_to_root("tests/data/demo-resolver/"),
            "targets": [
                {
                    "identifier": "sexontology",
                    "bucket": "sex.json"
                }
            ],
            "result_resource_mapping": full_path_relative_to_root(
                "examples/configurations/demo-resolver/term-to-resource-mapping.hjson"
            )
        }
    ],
    "ontology": [
        {
            "resolver": "DemoResolver",
            "origin": "directory",
            "source": full_path_relative_to_root("tests/data/demo-resolver/"),
            "targets": [
                {
                    "identifier": "cells",
                    "bucket": "cell_types.json"
                }
            ],
            "result_resource_mapping": full_path_relative_to_root(
                "examples/configurations/demo-resolver/term-to-resource-mapping.hjson"
            )
        }
    ],
    "entities": [
        {
            "resolver": "DemoResolver",
            "origin": "directory",
            "source": full_path_relative_to_root("tests/data/demo-resolver/"),
            "targets": [
                {
                    "identifier": "agents",
                    "bucket": "agents.json"
                }
            ],
            "result_resource_mapping": full_path_relative_to_root(
                "examples/configurations/demo-resolver/entity-to-resource-mapping.hjson"
            )
        }
    ],
    "schemaorg": [
        {
            "resolver": "EntityLinkerSkLearn from kgentitylinkingsklearn",
            "origin": "directory",
            "source": full_path_relative_to_root("examples/data/"),
            "targets": [
                {
                    "identifier": "terms",
                    "bucket": "tfidfvectorizer_model_schemaorg_linking"
                }
            ],
            "result_resource_mapping": full_path_relative_to_root(
                "examples/configurations/entitylinking-resolver/entitylinking-mapper.hjson"
            )
        }
    ]
}

Configure formatters¶

In [ ]:

config["Formatters"] = {
    "identifier": "https://kg.example.ch/{}/{}",
}

Save configuration¶

In [ ]:

import yaml

In [ ]:

with open(full_path_relative_to_root("examples/configurations/forge.yml"), "w") as f:
    yaml.dump(config, f)