FREYA Logo FREYA WP2 User Story 10 As a funder, we want to be able to find all the outputs related to our awarded grants, including block grants such as doctoral training grants, for management info and looking at impact.

Funders are interested in monitoring the output of grants they award - while the grant is active as well as retrospectively. The quality, quantity and types of the grant's outputs are useful proxies for the value obtained as a result of the funder's investment.

This notebook uses the DataCite GraphQL API to retrieve all outputs of FREYA grant award from European Union to date.

Goal: By the end of this notebook you should be able to:

  • Retrieve all outputs of a grant award from a specific funder;
  • Plot number of outputs per year-quarter of the grant award duration;
  • Display de-duplicated outputs in tabular format, including the number of their citations, views and downloads;
  • Plot a pie chart of the number of outputs per resource type;
  • Display an interactive chord plot of co-authorship relationships across all outputs, e.g.
  • Plot a pie chart of the number of outputs per license type;
  • Plot an interactive stacked bar plot showing the proportion of outputs of each type issued under a given license type.

Install libraries and prepare GraphQL client

In [29]:
%%capture
# Install required Python packages
!pip install gql requests chord==0.0.17 numpy
In [30]:
# Prepare the GraphQL client
import requests
from IPython.display import display, Markdown
from gql import gql, Client
from gql.transport.requests import RequestsHTTPTransport

_transport = RequestsHTTPTransport(
    url='https://api.datacite.org/graphql',
    use_json=True,
)

client = Client(
    transport=_transport,
    fetch_schema_from_transport=True,
)

Define and run GraphQL query

Define the GraphQL query to find all outputs of FREYA grant award from European Union to date.

In [31]:
# Generate the GraphQL query: find all outputs of FREYA grant award (https://cordis.europa.eu/project/id/777523) from funder (EU) to date
query_params = {
    "funderId" : "https://doi.org/10.13039/501100000780",
    "funderAwardQuery" : "fundingReferences.awardNumber:777523",
    "maxWorks" : 200
}

query = gql("""query getGrantOutputsForFunderAndAward($funderId: ID!, $funderAwardQuery: String!, $maxWorks: Int!)
{
funder(id: $funderId) {
  name
  works(query: $funderAwardQuery, first: $maxWorks) {
      totalCount
      nodes {
        id
        formattedCitation(style: "vancouver")
        titles {
          title
        }
        descriptions {
          description
        }        
        types {
          resourceType
        }
        dates {
          date
          dateType
        }
        versionOfCount
        rights {
          rights
          rightsIdentifier
          rightsUri
        }        
        creators {
          id
          name
        }
        fundingReferences {
          funderIdentifier
          funderName
          awardNumber
          awardTitle
        }
        citationCount
        viewCount
        downloadCount
      }
    }
  }
}
""")

Run the above query via the GraphQL client

In [32]:
import json
data = client.execute(query, variable_values=json.dumps(query_params))

Display total number of works

Display the total number of FREYA grant award outputs to date.

In [33]:
# Get the total number of outputs to date
funder = data['funder']['works']
display(Markdown(str(funder['totalCount'])))

92

Plot number of works per quarter

Display a bar plot of number of FREYA grant award outputs to date, per each quarter of project's duration.

In [34]:
# Plot the number of FREYA outputs to date, by year
import matplotlib.pyplot as plt
from matplotlib.ticker import FormatStrFormatter
import numpy as np

# Return quarter (number) given month (number)
def get_quarter(month):
    return (month - 1) // 3 + 1

# Return list of consecutive years between min_year_quarter and max_year_quarter inclusive
def get_consecutive_year_quarters(min_year_quarter, max_year_quarter):
    year_quarters = ["%d Q%d" % (min_year_quarter[0],min_year_quarter[1])]
    yq = min_year_quarter
    while yq != max_year_quarter:
        year = yq[0]
        quarter = yq[1]
        if quarter == 4:
            year += 1
            quarter = 1
        else:
            quarter += 1
        yq = (year, quarter)
        year_quarters.append("%d Q%d" % (year,quarter))
    year_quarters.append("%d Q%d" % (max_year_quarter[0],max_year_quarter[1])) 
    return year_quarters

plt.rcdefaults()

# Retrieve works counts by year-quarter from nodes
# Pick out date of type: 'Issued'; failing that use 'Created' date.
num_outputs_dict = {}
funder = data['funder']['works']
for r in funder['nodes']:
    node_date = None
    for date_dict in r['dates']:
        ym = date_dict['date'].split('-')[0:2]
        if len(ym) < 2:
            continue
        yq = ym[0] + " Q" + str(get_quarter(int(ym[1])))
        if node_date is None:
            if date_dict['dateType'] in ['Issued', 'Created']:
                node_date = yq
        else:
            if date_dict['dateType'] in ['Issued']:
                node_date = yq
    if node_date:
        if node_date not in num_outputs_dict:
            num_outputs_dict[node_date] = 0
        num_outputs_dict[node_date] += 1;
    
# Sort works counts by year-quarter in chronological order
sorted_year_quarters = sorted(list(num_outputs_dict.keys()))
# Get all consecutive year-quarters FREYA-specific start-end year-quarter
year_quarters = get_consecutive_year_quarters((2017,4), (2020,4))       
# Populate non-zero counts for year_quarters                          
num_outputs = []
for yq in year_quarters:
    if yq in sorted_year_quarters:
        num_outputs.append(num_outputs_dict[yq])
    else:
        num_outputs.append(0)
        
# Generate a plot of number of grant outputs by year - quarter
fig, ax = plt.subplots(1, 1, figsize = (10, 5))
x_pos = np.arange(len(year_quarters))
ax.bar(x_pos, num_outputs, align='center', color='blue', edgecolor='black', linewidth = 0.1, alpha=0.5)
ax.set_xticks(x_pos)
ax.set_xticklabels(year_quarters, rotation='vertical')
ax.set_ylabel('Number of outputs')
ax.set_xlabel('Year Quarter')
ax.set_title('Number of Grant Award Outputs per Year-Quarter')
plt.show()

Display de-duplicated works in tabular format

Display the outputs of FREYA grant award in a html table, including the number of their citations, views and downloads. Note that the outputs are de-duplicated, i.e. outputs that are versions of another output are excluded.

In [35]:
from IPython.core.display import display, HTML
import textwrap
xstr = lambda s: 'General' if s is None else str(s)

# Get details for each output
outputs = [['ID','Type','Publication Date','Formatted Citation','Descriptions', 'Number of Citations', 'Number of Views', 'Number of Downloads']]
# Since there is scope for duplicates in Zenodo, versions of previously seen nodes are considered duplicates and stored in duplicate_versions so that 
# they can be excluded if seen later
for r in funder['nodes']:
    id = '<a href="%s">%s</a></html>' % (r['id'], '/'.join(r['id'].split("/")[3:]))
    if r['versionOfCount'] > 0:
        # If the current output is a version of another one, exclude it
        continue
    # As Publication Date, pick out date of type: 'Issued'; failing that use 'Created' date.
    pub_date = None
    for date_dict in r['dates']:
        if pub_date is None:
            if date_dict['dateType'] in ['Issued', 'Created']:
                pub_date = date_dict['date'];
        else:
            if date_dict['dateType'] in ['Issued']:
                pub_date = date_dict['date'];
    
    titles = '; '.join([s['title'] for s in r['titles']])
    creators = '; '.join(['<a href="%s">%s</a>' % (s['id'],s['name']) for s in r['creators']])
    formatted_citation = "%s. %s. %s; Available from: %s" % (creators, titles, pub_date, id) 
    resource_type = xstr(r['types']['resourceType'])
    descriptions = textwrap.shorten('; '.join([s['description'] for s in r['descriptions']]), width=200, placeholder="...")
    output = [id, resource_type, pub_date, formatted_citation, descriptions, str(r['citationCount']), str(r['viewCount']), str(r['downloadCount'])]
    outputs += [output]
    
# Display outputs as html table 
html_table = '<html><table>'   
html_table += '<tr><th style="text-align:center;">' + '</th><th style="text-align:center;">'.join(outputs[0]) + '</th></tr>'
for row in outputs[1:]:
    html_table += '<tr><td style="text-align:left;">' + '</td><td style="text-align:left;">'.join(row) + '</td></tr>'
html_table += '</table></html>'
display(HTML(html_table))
IDTypePublication DateFormatted CitationDescriptionsNumber of CitationsNumber of ViewsNumber of Downloads
10.5281/zenodo.1202173General2018-03-17Fenner, Martin; Crosas, Merce; Durand, Gustavo; Wimalaratne, Sarala; Gräf, Florian; Hallett, Richard; Bernal Llinares, Manuel; Schindler, Uwe; Clark, Tim. Listing of data repositories that embed schema.org metadata in dataset landing pages. 2018-03-17; Available from: 10.5281/zenodo.1202173Machine-readable metadata available from landing pages for datasets facilitate data citation by enabling easy integration with reference managers and other tools used in a data citation workflow....000
10.5281/zenodo.1263942General2018-03-17Fenner, Martin; Crosas, Merce; Durand, Gustavo; Wimalaratne, Sarala; Gräf, Florian; Hallett, Richard; Bernal Llinares, Manuel; Schindler, Uwe; Clark, Tim. Listing of data repositories that embed schema.org metadata in dataset landing pages. 2018-03-17; Available from: 10.5281/zenodo.1263942Machine-readable metadata available from landing pages for datasets facilitate data citation by enabling easy integration with reference managers and other tools used in a data citation workflow....200
10.5281/zenodo.2548715Presentation2019-01-24Simons, Natasha; Brown, Christopher; Braukmann, Ricarda. Adventure Time: the PID community challenge - Presentation at PIDapalooza19. 2019-01-24; Available from: 10.5281/zenodo.2548715Presentation of the "Adventure Time: the PID community challenge" - session at PIDapalooza19. Abstract of the session: PIDs are a core component of open research across the global scholarly...000
10.5281/zenodo.2548714Presentation2019-01-24Simons, Natasha; Brown, Christopher; Braukmann, Ricarda. Adventure Time: the PID community challenge - Presentation at PIDapalooza19. 2019-01-24; Available from: 10.5281/zenodo.2548714Presentation of the "Adventure Time: the PID community challenge" - session at PIDapalooza19. Abstract of the session: PIDs are a core component of open research across the global scholarly...000
10.5281/zenodo.2549227Presentation2019-01-25Reimer, Torsten; Madden, Frances. From Standard to Community Resource: A View on ISNIs and ORG IDs. 2019-01-25; Available from: 10.5281/zenodo.2549227Over the last year, the International Standard Name Identifier board have been considering the ways in which ISNI as a system can improve to meet new challenges and become more open and...000
10.5281/zenodo.2552138Presentation2019-01-29Baars, Chris; De Jong, Maaike. NARCIS & FREYA Can PIDs contribute to research in context?. 2019-01-29; Available from: 10.5281/zenodo.2552138Abstract: NARCIS, the national gateway to scholarly information in the Netherlands (www.narcis.nl) contains information about publications, datasets, research projects, researchers and...000
10.5281/zenodo.2552137Presentation2019-01-29Baars, Chris; De Jong, Maaike. NARCIS & FREYA Can PIDs contribute to research in context?. 2019-01-29; Available from: 10.5281/zenodo.2552137Abstract: NARCIS, the national gateway to scholarly information in the Netherlands (www.narcis.nl) contains information about publications, datasets, research projects, researchers and...000
10.5438/jwvf-8a66BlogPosting2019Fenner, Martin; Aryani, Amir. Introducing the PID Graph. 2019; Available from: 10.5438/jwvf-8a66This post has been cross-posted from the FREYA blog. Persistent identifiers (PIDs) are not only important to uniquely identify a publication, dataset, or person, but the metadata for these...400
10.5438/wy92-xj57BlogPosting2019-04-10Fenner, Martin. Exposing DOI metadata provenance. 2019-04-10; Available from: 10.5438/wy92-xj57DOI metadata provenance is describing the history of a particular DOI metadata record, i.e. what changes were made when and by whom. This information is now stored and provided via an API for all...000
10.5281/zenodo.2649230Report2019-04-23Ferguson, Christine; McEntyre, Johanna; Hendricks, Ginny; Dohna, Tina; Koop-Jakobsen, Ketil; Madden, Frances; Dallmeier-Tiessen, Sunje; Sandt, Stephanie Van De; Lavasa, Artemis; Lambert, Simon; Bunakov, Vasily; Dasler, Robin; Fenner, Martin. Deliverable D3.2 Requirements for Selected New PID Services. 2019-04-23; Available from: 10.5281/zenodo.2649230A comprehensive analysis of user stories relating to a range of entities needing persistent identifiers, with conclusions for further work in the FREYA project.000
10.5281/zenodo.2649229Report2019-04-28Ferguson, Christine; McEntyre, Johanna; Hendricks, Ginny; Dohna, Tina; Koop-Jakobsen, Ketil; Madden, Frances; Dallmeier-Tiessen, Sunje; Sandt, Stephanie Van De; Lavasa, Artemis; Lambert, Simon; Bunakov, Vasily; Dasler, Robin; Fenner, Martin. Deliverable D3.2 Requirements for Selected New PID Services. 2019-04-28; Available from: 10.5281/zenodo.2649229A comprehensive analysis of user stories relating to a range of entities needing persistent identifiers, with conclusions for further work in the FREYA project. The deliverable was updated after...000
10.5438/qab1-n315BlogPosting2019Fenner, Martin. The DataCite GraphQL API is now open for (pre-release) business. 2019; Available from: 10.5438/qab1-n315DataCite DOIs describe resources such as datasets, samples, software and publications with rich metadata. An important part of this metadata is the description of connections between resources that...300
10.5281/zenodo.2866420Presentation2019-05-14Chue Hong, Neil; Cope, Jez; Madden, Frances. Setting the Scene to Cite Software. 2019-05-14; Available from: 10.5281/zenodo.2866420Introduction to Day 2 of the Software Citation Workshop organised by the Software Sustainability Institute, the British Library and The Alan Turing Institute.000
10.5281/zenodo.2866419Presentation2019-05-14Chue Hong, Neil; Cope, Jez; Madden, Frances. Setting the Scene to Cite Software. 2019-05-14; Available from: 10.5281/zenodo.2866419Introduction to Day 2 of the Software Citation Workshop organised by the Software Sustainability Institute, the British Library and The Alan Turing Institute.000
10.5438/hwaw-xe52BlogPosting2019Fenner, Martin. Using Jupyter Notebooks with GraphQL and the PID Graph. 2019; Available from: 10.5438/hwaw-xe52Two weeks ago DataCite announced the pre-release version of a GraphQL API [Fenner (2019)]. GraphQL simplifies complex queries that for example want to retrieve information about the authors,...200
10.5281/zenodo.3248653Project deliverable2019-06-18Fenner, Martin; Wass, Joe; Demeranville, Tom; Wimalaratne, Sarala; Hallett, Richard. D2.2 PID Metadata Provenance. 2019-06-18; Available from: 10.5281/zenodo.3248653The main focus of this document is describing the implementations of provenance tracking by the persistent identifier (PID) providers Crossref, ORCID, Identifiers.org (EMBL-EBI), and DataCite in...000
10.5281/zenodo.3248652Project deliverable2019-06-18Fenner, Martin; Wass, Joe; Demeranville, Tom; Wimalaratne, Sarala; Hallett, Richard. D2.2 PID Metadata Provenance. 2019-06-18; Available from: 10.5281/zenodo.3248652The main focus of this document is describing the implementations of provenance tracking by the persistent identifier (PID) providers Crossref, ORCID, Identifiers.org (EMBL-EBI), and DataCite in...100
10.5281/zenodo.3249833Project deliverable2019-06-19Lavasa, Artemis; Dallmeier-Tiessen, Sünje; Sandt, Stephanie Van De; Dohna, Tina; Koop-Jakobsen, Ketil; Schindler, Uwe; Ferguson, Christine; McEntyre, Johanna; Madden, Frances; Lambert, Simon; Bunakov, Vasily; Baars, Chris. D4.2 Using the PID Graph: Provenance in Disciplinary Systems. 2019-06-19; Available from: 10.5281/zenodo.3249833The main focus of this deliverable is the different approaches to provenance as understood, expressed, and implemented by the FREYA disciplinary partners in their various pilot applications. The...000
10.5281/zenodo.3249832Project deliverable2019-06-19Lavasa, Artemis; Dallmeier-Tiessen, Sünje; Sandt, Stephanie Van De; Dohna, Tina; Koop-Jakobsen, Ketil; Schindler, Uwe; Ferguson, Christine; McEntyre, Johanna; Madden, Frances; Lambert, Simon; Bunakov, Vasily; Baars, Chris. D4.2 Using the PID Graph: Provenance in Disciplinary Systems. 2019-06-19; Available from: 10.5281/zenodo.3249832The main focus of this deliverable is the different approaches to provenance as understood, expressed, and implemented by the FREYA disciplinary partners in their various pilot applications. The...000
10.5281/zenodo.3250702Project deliverable2019-06-20Lambert, Simon. D6.1 First Annual Report on PID Commons and Sustainability. 2019-06-20; Available from: 10.5281/zenodo.3250702The first report on FREYA’s sustainability takes a wide view of the meaning and requirements on sustainability that the context of the project imply. The “what”, “who” and “how” of sustainability...000
10.5281/zenodo.3250701Project deliverable2019-06-20Lambert, Simon. D6.1 First Annual Report on PID Commons and Sustainability. 2019-06-20; Available from: 10.5281/zenodo.3250701The first report on FREYA’s sustainability takes a wide view of the meaning and requirements on sustainability that the context of the project imply. The “what”, “who” and “how” of sustainability...000
10.14454/3bpw-w381Computational Notebook2019-06-30Fenner, Martin. Jupyter Notebook FREYA PID Graph Key Performance Indicators (KPIs). 2019-06-30; Available from: 10.14454/3bpw-w381200
10.5438/bv9z-dc66BlogPosting2019-07-01Fenner, Martin. Tracking the Growth of the PID Graph. 2019-07-01; Available from: 10.5438/bv9z-dc66The connections between scholarly resources generated by persistent identifiers (PIDs) and associated metadata form a graph: the PID Graph [Fenner & Aryani (2019)]. We developed this PID Graph...000
10.5281/zenodo.3361391Presentation2019-07-16Madden, Frances. FREYA RDA UK Workshop July 2019. 2019-07-16; Available from: 10.5281/zenodo.3361391Presentation introducing the FREYA project at a joint RDA UK and FREYA workshop held 16 July 2019.000
10.5281/zenodo.3361390Presentation2019-07-16Madden, Frances. FREYA RDA UK Workshop July 2019. 2019-07-16; Available from: 10.5281/zenodo.3361390Presentation introducing the FREYA project at a joint RDA UK and FREYA workshop held 16 July 2019.000
10.5281/zenodo.3361397Presentation2019-06-26Madden, Frances. The Power of PIDs. 2019-06-26; Available from: 10.5281/zenodo.3361397Lightning Talk slide presented at Carpentry Connect 2019 in Manchester UK.000
10.5281/zenodo.3361396Presentation2019-06-26Madden, Frances. The Power of PIDs. 2019-06-26; Available from: 10.5281/zenodo.3361396Lightning Talk slide presented at Carpentry Connect 2019 in Manchester UK.000
10.5281/zenodo.3361429Presentation2019-07-16Lammey, Rachael. Embracing funder workflows (or Grant why-ds…). 2019-07-16; Available from: 10.5281/zenodo.3361429Presentation on how Crossref are starting to implement persistent identifiers for grants. Presented at RDA UK FREYA workshop,16 July 2019.000
10.5281/zenodo.3361430Presentation2019-07-16Lammey, Rachael. Embracing funder workflows (or Grant why-ds…). 2019-07-16; Available from: 10.5281/zenodo.3361430Presentation on how Crossref are starting to implement persistent identifiers for grants. Presented at RDA UK FREYA workshop,16 July 2019.000
10.5281/zenodo.3361436Presentation2019-07-16Ferguson, Christine. Preprints and literature provenance in Europe PMC. 2019-07-16; Available from: 10.5281/zenodo.3361436Presentation on pre-print versioning from Europe PMC given at the FREYA RDA UK Workshop, 16 July 2019.000
10.5281/zenodo.3361435Presentation2019-07-16Ferguson, Christine. Preprints and literature provenance in Europe PMC. 2019-07-16; Available from: 10.5281/zenodo.3361435Presentation on pre-print versioning from Europe PMC given at the FREYA RDA UK Workshop, 16 July 2019.000
10.5281/zenodo.3361512Presentation2019-07-16Brown, Christopher. RDA Europe Update. 2019-07-16; Available from: 10.5281/zenodo.3361512Update presentation on the activities of RDA Europe from the RDA UK FREYA workshop on 16 July 2019.000
10.5281/zenodo.3361511Presentation2019-07-16Brown, Christopher. RDA Europe Update. 2019-07-16; Available from: 10.5281/zenodo.3361511Update presentation on the activities of RDA Europe from the RDA UK FREYA workshop on 16 July 2019.000
10.5281/zenodo.3361399Presentation2019-07-16Brown, Christopher. JISC and PIDs. 2019-07-16; Available from: 10.5281/zenodo.3361399Presentation outlining JISC's work relating to different persistent identifiers. Presented at the joint FREYA RDA UK Node Workshop 16 July 2019.000
10.5281/zenodo.3361398Presentation2019-07-16Brown, Christopher. JISC and PIDs. 2019-07-16; Available from: 10.5281/zenodo.3361398Presentation outlining JISC's work relating to different persistent identifiers. Presented at the joint FREYA RDA UK Node Workshop 16 July 2019.000
10.5281/zenodo.3361420Presentation2019-07-16Bunakov, Vasily. STFC case study: PhD research graph. 2019-07-16; Available from: 10.5281/zenodo.3361420Presentation on an STFC example of a graph created to discover relationships between PhD metadata. Presented at the RDA UK FREYA workshop, 16 July 2019.000
10.5281/zenodo.3361419Presentation2019-07-16Bunakov, Vasily. STFC case study: PhD research graph. 2019-07-16; Available from: 10.5281/zenodo.3361419Presentation on an STFC example of a graph created to discover relationships between PhD metadata. Presented at the RDA UK FREYA workshop, 16 July 2019.000
10.5281/zenodo.3361404Presentation2019-07-16Fenner, Martin. RDA Open Science Graphs for FAIR Data Interest Group. 2019-07-16; Available from: 10.5281/zenodo.3361404Presentation on the proposed RDA Interest Group Open Science Graphs for FAIR Data. Presented at the RDA UK Node FREYA workshop on 16 July 2019.000
10.5281/zenodo.3361405Presentation2019-07-16Fenner, Martin. RDA Open Science Graphs for FAIR Data Interest Group. 2019-07-16; Available from: 10.5281/zenodo.3361405Presentation on the proposed RDA Interest Group Open Science Graphs for FAIR Data. Presented at the RDA UK Node FREYA workshop on 16 July 2019.000
10.5281/zenodo.3361407Presentation2019-07-16Cope, Jez. DataCite in the UK. 2019-07-16; Available from: 10.5281/zenodo.3361407Presentation on DataCite services in the UK provided by the British Library. Presented at a joint RDA UK node FREYA workshop on 16 July 2019.000
10.5281/zenodo.3361406Presentation2019-07-16Cope, Jez. DataCite in the UK. 2019-07-16; Available from: 10.5281/zenodo.3361406Presentation on DataCite services in the UK provided by the British Library. Presented at a joint RDA UK node FREYA workshop on 16 July 2019.000
10.5281/zenodo.3361413Presentation2019-07-16Demeranville, Tom. ORCID Update - RDA UK/FREYA Workshop. 2019-07-16; Available from: 10.5281/zenodo.3361413Presentation on ORCID developments in 2019 from the RDA UK/FREYA Workshop in July 2019.000
10.5281/zenodo.3361412Presentation2019-07-16Demeranville, Tom. ORCID Update - RDA UK/FREYA Workshop. 2019-07-16; Available from: 10.5281/zenodo.3361412Presentation on ORCID developments in 2019 from the RDA UK/FREYA Workshop in July 2019.000
10.5281/zenodo.3462141Project deliverable2019-08-31Madden, Frances; Mayville, Louise; Cousijn, Helena; Horik, René Van; Kotarski, Rachael. Deliverable 5.4 Initial Training Materials. 2019-08-31; Available from: 10.5281/zenodo.3462141This interim report describes the approach taken to creating training materials in the FREYA project and provides an overview of the materials created and links to them. Up to this point in the...000
10.5281/zenodo.3462140Project deliverable2019-08-31Madden, Frances; Mayville, Louise; Cousijn, Helena; Horik, René Van; Kotarski, Rachael. Deliverable 5.4 Initial Training Materials. 2019-08-31; Available from: 10.5281/zenodo.3462140This interim report describes the approach taken to creating training materials in the FREYA project and provides an overview of the materials created and links to them. Up to this point in the...000
10.5281/zenodo.3460655General2019-09-25Madden, Frances; Kearney, Nicole; Boruta, Luc; Marín-Arraiza, Paloma. Fourth FREYA Ambassador Webinar. 2019-09-25; Available from: 10.5281/zenodo.3460655Agenda Nicole Kearney, Biodiversity Heritage Library - What are we ‘DOI’ng about the out-of-copyright literature Luc Boruta, Thunken - Cobaltmetrics: Better a URL Today Than a PID Tomorrow Paloma...000
10.5281/zenodo.3460654General2019-09-25Madden, Frances; Kearney, Nicole; Boruta, Luc; Marín-Arraiza, Paloma. Fourth FREYA Ambassador Webinar. 2019-09-25; Available from: 10.5281/zenodo.3460654Agenda Nicole Kearney, Biodiversity Heritage Library - What are we ‘DOI’ng about the out-of-copyright literature Luc Boruta, Thunken - Cobaltmetrics: Better a URL Today Than a PID Tomorrow Paloma...000
10.5281/zenodo.3514573Presentation2019-10-21Ferguson, Christine; Fenner, Martin. The FREYA project: Collaborating to link people, papers, data, to new things…. 2019-10-21; Available from: 10.5281/zenodo.3514573Effective scholarly research depends on collecting and linking accurate data about the varied components and outputs. Persistent identifiers (PIDs) such as ORCID IDs, DOIs, or accession...000
10.5281/zenodo.3514572Presentation2019-10-21Ferguson, Christine; Fenner, Martin. The FREYA project: Collaborating to link people, papers, data, to new things…. 2019-10-21; Available from: 10.5281/zenodo.3514572Effective scholarly research depends on collecting and linking accurate data about the varied components and outputs. Persistent identifiers (PIDs) such as ORCID IDs, DOIs, or accession...000
10.5281/zenodo.3517853Presentation2019-10-24de Jong, Maaike; Cousijn, Helena; Lambert, Simon; Madden, Frances; Dohna, Tina; Koop-Jakobsen, Ketil; Garza, Kristian. Project FREYA: Connecting Open Knowledge in the European Open Science Cloud - Materials. 2019-10-24; Available from: 10.5281/zenodo.3517853Materials from the FREYA event Project FREYA: Connecting Open Knowledge in the European Open Science Cloud held on 21 October 2019 co-located with RDA P14 in Helsinki, Finland. Materials include...000
10.5281/zenodo.3517852Presentation2019-10-24de Jong, Maaike; Cousijn, Helena; Lambert, Simon; Madden, Frances; Dohna, Tina; Koop-Jakobsen, Ketil; Garza, Kristian. Project FREYA: Connecting Open Knowledge in the European Open Science Cloud - Materials. 2019-10-24; Available from: 10.5281/zenodo.3517852Materials from the FREYA event Project FREYA: Connecting Open Knowledge in the European Open Science Cloud held on 21 October 2019 co-located with RDA P14 in Helsinki, Finland. Materials include...000
10.5281/zenodo.3554196Report2019-04-28Ferguson, Christine; McEntyre, Johanna; Hendricks, Ginny; Dohna, Tina; Koop-Jakobsen, Ketil; Madden, Frances; Dallmeier-Tiessen, Sunje; Sandt, Stephanie Van De; Lavasa, Artemis; Lambert, Simon; Bunakov, Vasily; Dasler, Robin; Fenner, Martin. Deliverable D3.2 Requirements for Selected New PID Services. 2019-04-28; Available from: 10.5281/zenodo.3554196A comprehensive analysis of user stories relating to a range of entities needing persistent identifiers, with conclusions for further work in the FREYA project. The deliverable was updated after...000
10.5281/zenodo.3554255Project deliverable2019-10-18Ferguson, Christine; McEntrye, Jo; Bunakov, Vasily; Lambert, Simon; Sandt, Stephanie Van Der; Kotarski, Rachael; Stewart, Sarah; MacEwan, Andrew; Fenner, Martin; Cruse, Patricia; Horik, René Van; Dohna, Tina; Koop-Jacobsen, Ketil; Schindler, Uwe; McCafferty, Siobhan. D3.1Survey of Current PID Services Landscape - Revised. 2019-10-18; Available from: 10.5281/zenodo.3554255A comprehensive survey of the landscape of persistent identifiers across many disciplines is presented, with assessments of maturity of different PID types and conclusions for the future. [Revised...000
10.5281/zenodo.3554254Project deliverable2019-10-18Ferguson, Christine; McEntrye, Jo; Bunakov, Vasily; Lambert, Simon; Sandt, Stephanie Van Der; Kotarski, Rachael; Stewart, Sarah; MacEwan, Andrew; Fenner, Martin; Cruse, Patricia; Horik, René Van; Dohna, Tina; Koop-Jacobsen, Ketil; Schindler, Uwe; McCafferty, Siobhan. D3.1Survey of Current PID Services Landscape - Revised. 2019-10-18; Available from: 10.5281/zenodo.3554254A comprehensive survey of the landscape of persistent identifiers across many disciplines is presented, with assessments of maturity of different PID types and conclusions for the future. [Revised...000
10.5281/zenodo.3564210Presentation2019-12-05Morselli, Francesca; Madden, Frances; Burton, Kath; Blaney, Jonathan. (Effectively) Communicating your research online presentations. 2019-12-05; Available from: 10.5281/zenodo.3564210Presentations from the event (Effectively) Communicating your research online, held on 4 December 2019 at the Foundling Museum in London, UK. The event, aimed at early career researchers and PhD...000
10.5281/zenodo.3564209Presentation2019-12-05Morselli, Francesca; Madden, Frances; Burton, Kath; Blaney, Jonathan. (Effectively) Communicating your research online presentations. 2019-12-05; Available from: 10.5281/zenodo.3564209Presentations from the event (Effectively) Communicating your research online, held on 4 December 2019 at the Foundling Museum in London, UK. The event, aimed at early career researchers and PhD...000
10.5281/zenodo.3585742Preprint2019-12-19Bangert, Daniel; Hermans, Emilie; van Horik, René; de Jong, Maaike; Koers, Hylke; Mokrane, Mustapha. Recommendations for Services in a FAIR data ecosystem. 2019-12-19; Available from: 10.5281/zenodo.3585742This report highlights common challenges and priorities, and proposes a set of initial recommendations on how existing data infrastructures can evolve and collaborate to provide services that...000
10.5281/zenodo.3585741Preprint2019-12-19Bangert, Daniel; Hermans, Emilie; van Horik, René; de Jong, Maaike; Koers, Hylke; Mokrane, Mustapha. Recommendations for Services in a FAIR data ecosystem. 2019-12-19; Available from: 10.5281/zenodo.3585741This report highlights common challenges and priorities, and proposes a set of initial recommendations on how existing data infrastructures can evolve and collaborate to provide services that...000
10.5281/zenodo.3631166Presentation2020-01-30Braukmann, Ricarda; Cousijn, Helena; Lammey, Rachael; Madden, Frances; Meadows, Alice. PIDforum.org - A global discussion platform about PIDs. 2020-01-30; Available from: 10.5281/zenodo.3631166Presentation and discussion on pidforum.org - the global discussion platform for all things PID-related held at PIDapalooza 2020.000
10.5281/zenodo.3631167Presentation2020-01-30Braukmann, Ricarda; Cousijn, Helena; Lammey, Rachael; Madden, Frances; Meadows, Alice. PIDforum.org - A global discussion platform about PIDs. 2020-01-30; Available from: 10.5281/zenodo.3631167Presentation and discussion on pidforum.org - the global discussion platform for all things PID-related held at PIDapalooza 2020.000
10.5281/zenodo.3632730Presentation2020-01-31Ferguson, Christine. Making connections beyond papers, people, data (www.europepmc.org). 2020-01-31; Available from: 10.5281/zenodo.3632730A talk demonstrating a selection PID services at Europe PMC that connect up resources in new ways and link to resources that are currently emerging with their own bespoke PIDs such as grants and...000
10.5281/zenodo.3632729Presentation2020-01-31Ferguson, Christine. Making connections beyond papers, people, data (www.europepmc.org). 2020-01-31; Available from: 10.5281/zenodo.3632729A talk demonstrating a selection PID services at Europe PMC that connect up resources in new ways and link to resources that are currently emerging with their own bespoke PIDs such as grants and...000
10.5281/zenodo.3674733Presentation2020-02-19Madden, Frances; Cope, Jez; Ashton, Jan. Building the Picture behind a Dataset. 2020-02-19; Available from: 10.5281/zenodo.3674733Slides from a presentation at IDCC20, Building the Picture behind a Dataset, which describes an experiment to enhance provenance information of datasets using persistent identifiers in the...000
10.5281/zenodo.3759090Presentation2020-04-21Braukmann, Ricarda; Dasler, Robin; Koop-Jakobsen, Ketil. RDC Webinar: The PID Graph and its Potential: Extending PIDs for Research Data - Presentations. 2020-04-21; Available from: 10.5281/zenodo.3759090The Persistent Identifiers (PID) Graph provides a great example of the potential of persistent identifiers. Developed through the FREYA project, the PID Graph provides a way to connect and...000
10.5281/zenodo.3759089Presentation2020-04-21Braukmann, Ricarda; Dasler, Robin; Koop-Jakobsen, Ketil. RDC Webinar: The PID Graph and its Potential: Extending PIDs for Research Data - Presentations. 2020-04-21; Available from: 10.5281/zenodo.3759089The Persistent Identifiers (PID) Graph provides a great example of the potential of persistent identifiers. Developed through the FREYA project, the PID Graph provides a way to connect and...000
10.5438/yfck-mv39BlogPosting2020-05-06Fenner, Martin. Powering the PID Graph: announcing the DataCite GraphQL API. 2020-05-06; Available from: 10.5438/yfck-mv39Today DataCite launches a new API that powers the PID Graph, the graph formed by scholarly resources described by persistent identifiers (PIDs) and the connections between them. The API is powered...000
10.5281/zenodo.3801815Poster2020-05-06Braukmann, Ricarda. The PID Forum. 2020-05-06; Available from: 10.5281/zenodo.3801815This poster presents pidforum.org, a global information and discussion platform about persistent identifiers (PIDs).000
10.5281/zenodo.3817563General2020-05-08Dasler, Robin. FREYA PID Graph Demo. 2020-05-08; Available from: 10.5281/zenodo.3817563Introductory video on the PID Graph being developed by the FREYA project. Submitted for the EOSC-hub Week 2020.000
10.5281/zenodo.3862655General2020-05-28Madden, Frances; van Horik, René; van de Sandt, Stephanie; Lavasa, Artemis; Cousijn, Helena. Guides to Choosing Persistent Identifiers - Version 1. 2020-05-28; Available from: 10.5281/zenodo.3862655The FREYA Project has compiled short guides to help with choosing persistent identifiers for various types of entities. These first versions have been released in May 2020 for community feedback...000
10.5281/zenodo.3875046Presentation2020-02-18, Dasler. Harnessing the Power of the PID Graph. 2020-02-18; Available from: 10.5281/zenodo.3875046Demonstration of the FREYA PID Graph and the DataCite GraphQL API that were presented at IDCC2020 in the "Data publishing & preservation demonstrations" session.000
10.5281/zenodo.3876248Poster2020-05-18Llinares, Bernal; Gómez, Ferrer; , Wimalaratne; Juty; , Hermjakob. Compact Identifiers Service in the Cloud. 2020-05-18; Available from: 10.5281/zenodo.3876248Identifiers.org has established itself as a stable system for identification and citation of life science data, using Persistent Identifiers (PIDs). Identifiers.org not only enables researchers to...000
10.25495/yw94-sp56PID Service2020DataCite. PID Services Registry. 2020; Available from: 10.25495/yw94-sp56This registry provides an overview of services related to Persistent Identifiers (PIDs). The PID Services Registry is maintained by DataCite and was developed within the European Commission-funded...000
10.25495/xkya-0g76PID Service2020DataCite. DataCite GraphQL API; Querying the PID Graph. 2020; Available from: 10.25495/xkya-0g76The DataCite GraphQL API support queries of the DataCite API using the GraphQL query language. DataCite DOIs describe resources such as datasets, samples, software, and publications with rich...000
10.25495/x8j5-z806PID Service2019DataCite. DataCite DOI Provenance API. 2019; Available from: 10.25495/x8j5-z806Metadata provenance is describing the history of a particular DataCite DOI metadata record, i.e. what changes were made when and by whom. This information is stored and provided via an API for all...000
10.25495/sh9r-dm82PID Service2005Alfred Wegener Institute For Polar And Marine Research; Marum. PANGAEA; Data Publisher for Earth & Environmental Science. 2005; Available from: 10.25495/sh9r-dm82The information system PANGAEA is operated as an Open Access library aimed at archiving, publishing and distributing georeferenced data from earth system research. The system guarantees long-term...000
10.25495/fxd6-ba56PID Service2019Data Archiving And Networked Services; British Library; DataCite. The PID Forum. 2019; Available from: 10.25495/fxd6-ba56The PID Forum is a global information and discussion platform about persistent identifiers (PIDs). The PID Forum aims to bring together the various communities working with PIDs in the research...000
10.14454/r0ed-fh20Computational Notebook2020-05-08Petryszak, Robert; Fenner, Martin; Lambert, Simon; Llinares, Manuel Bernal; Madden, Frances. FREYA PID Graph - Datacenter Publication Citations. 2020-05-08; Available from: 10.14454/r0ed-fh20Jupyter notebook that uses GraphQL to implement EC-funded FREYA Project User Story 1: As a data center, I want to see the citations of publications that use my repository for the underlying data,...000
10.14454/qaym-kt26Computational Notebook2020-05-10Petryszak, Robert; Fenner, Martin; Lambert, Simon; Llinares, Manuel Bernal; Madden, Frances. FREYA PID Graph - Grant Outputs. 2020-05-10; Available from: 10.14454/qaym-kt26Jupyter notebook that uses GraphQL to implement EC-funded FREYA Project User Story 10: As a funder, we want to be able to find all the outputs related to our awarded grants, including block grants...000
10.14454/27b7-9g84Computational Notebook2020-06-01Petryszak, Robert; Fenner, Martin; Lambert, Simon; Llinares, Manuel Bernal; Madden, Frances. FREYA PID Graph - Software Citations. 2020-06-01; Available from: 10.14454/27b7-9g84Jupyter notebook that uses GraphQL to implement EC-funded FREYA Project User Story 2: As a software author, I want to be able to see the citations of my software aggregated across all versions. so...000
10.14454/jkar-xj80Computational Notebook2020-05-19Petryszak, Robert; Fenner, Martin; Lambert, Simon; Llinares, Manuel Bernal; Madden, Frances. FREYA PID Graph - Dissertations. 2020-05-19; Available from: 10.14454/jkar-xj80Jupyter notebook that uses GraphQL to implement EC-funded FREYA Project User Story 5: As a student using the British Library's EThOS database, I want to be able to find all dissertations on a given...000
10.14454/62t3-0822Computational Notebook2020-05-25Petryszak, Robert; Fenner, Martin; Lambert, Simon; Llinares, Manuel Bernal; Madden, Frances. FREYA PID Graph - Researcher Co-authors. 2020-05-25; Available from: 10.14454/62t3-0822Jupyter notebook that uses GraphQL to implement EC-funded FREYA Project User Story 9: As a bibliometrician, I want to know all the co-authors of a particular researcher, so that I can do a network...000
10.14454/y785-xs19Computational Notebook2020-05-24Petryszak, Robert; Fenner, Martin; Lambert, Simon; Llinares, Manuel Bernal; Madden, Frances. FREYA PID Graph - Impacts of Data. 2020-05-24; Available from: 10.14454/y785-xs19Jupyter notebook that uses GraphQL to implement EC-funded FREYA Project User Story 8: As a longitudinal study, I want to be able to deduplicate the metrics/impact for our data, so that I can see...000
10.14454/fqq6-w751Collection2020-05-08Petryszak, Robert; Fenner, Martin; Lambert, Simon; Llinares, Manuel Bernal; Madden, Frances. FREYA PID Graph Jupyter Notebooks. 2020-05-08; Available from: 10.14454/fqq6-w751Jupyter notebooks that use GraphQL to implement EC-funded FREYA Project PID Graph user stories.000
10.14454/e0f7-zs08SoftwareSourceCode2020-06-26Hallett, Richard; Fenner, Martin; Fokianos, Pamfilos. Beagle: the code for the FREYA PID Services Registry. 2020-06-26; Available from: 10.14454/e0f7-zs08The FREYA project is building up human and technical resources to support persistent identifier (PID) infrastructure in the European Open Science Cloud (EOSC). One important gap identified during...000

Plot number of outputs per resource type

Plot as a pie chart the number of FREYA grant award outputs per resource type.

In [36]:
# Plot as a pie chart the number of outputs per resource type
import matplotlib.pyplot as plt
from matplotlib.ticker import FormatStrFormatter
import numpy as np
import operator

xstr = lambda s: 'General' if s is None else str(s)

plt.rcdefaults()

# Retrieve works counts by resource type from nodes
# Pick out date of type: 'Issued'; failing that use 'Created' date.
funder = data['funder']['works']
num_outputs_dict = {}
for r in funder['nodes']:
    resource_type = xstr(r['types']['resourceType'])
    if resource_type not in num_outputs_dict:
        num_outputs_dict[resource_type] = 0
    num_outputs_dict[resource_type] += 1;
    
# Sort resource types by count of work desc
sorted_num_outputs = sorted(num_outputs_dict.items(),key=operator.itemgetter(1),reverse=True)
# Populate lists needed for pie chart
resource_types = [s[0] for s in sorted_num_outputs] 
num_outputs = [s[1] for s in sorted_num_outputs] 

# Generate a pie chart of number of grant outputs by resource type
fig = plt.figure()
ax = fig.add_axes([0,0,1,1])
ax.set_title('Number of Grant Outputs per Resource Type')
ax.axis('equal')
ax.pie(num_outputs, labels = resource_types,autopct='%1.2f%%')
plt.show()

Display an interactive plot of co-authorship relationships across all outputs

Display an interactive chord plot representing co-authorship relationships across all FREYA grant award outputs.

In [40]:
# Generate a Chord plot representing co-authorship relationships across all grant award outputs
from chord import Chord       
from IPython.display import IFrame

all_creator_names_by_node = []
all_creator_names_set = set([])
funder = data['funder']['works']
for r in funder['nodes']:
    if r['versionOfCount'] > 0:
        # If the current output is a version of another one, exclude it
        continue
    # To minimise cropping of names in the below, retain just the first letter of the first name
    # if the author name is well formatted     
    creator_names = []
    for name in [s['name'] for s in r['creators'] if s['name']]:
        if name.find(",") > 0:
            creator_names.append(name[0:name.index(",") + 3])
        elif name.find(",") == 0:
            creator_names.append(name[1:].strip())
        else:
            creator_names.append(name)
    all_creator_names_by_node.append(creator_names)
    all_creator_names_set.update(creator_names)
    
# Assemble data structures for the co-authorship chord diagram
all_creator_names = sorted(list(all_creator_names_set))

# Initialise chord data matrix
length = len(all_creator_names)
coauthorship_matrix = []
for i in range(length):
    r = []
    for j in range(length):
        r.append(0)
    coauthorship_matrix.append(r)
    
# Populate chord data matrix
for node_creators in all_creator_names_by_node:
    for creator in node_creators:
        c_pos = all_creator_names.index(creator)
        for co_creator in node_creators:
            co_pos = all_creator_names.index(co_creator)
            if c_pos != co_pos:
                coauthorship_matrix[c_pos][co_pos] += 1

# display co-authorship cord diagram
plot = Chord(coauthorship_matrix, all_creator_names, padding=0.04, wrap_labels=False, margin=130, width=1000).to_html()
IFrame(src="./out.html", width=1000, height=1000)
Out[40]:

Plot number of outputs per license type

Plot as a pie chart the number of FREYA grant award outputs per license type.

In [38]:
# Plot as a pie chart the number of outputs per license type
import matplotlib.pyplot as plt
from matplotlib.ticker import FormatStrFormatter
import numpy as np
import operator

xstr = lambda s: 'General' if s is None else str(s)
xstr1 = lambda s: 'Not available' if s is None else str(s)

plt.rcdefaults()

# Retrieve works counts by resource type from nodes
funder = data['funder']
awardTitle = None

right2NumOutputs = {}
resourceType2NumOutputs = {}

for r in funder['works']['nodes']:
    resourceType = xstr(r['types']['resourceType'])
    if r['versionOfCount'] > 0:
        # If the current output is a version of another one, exclude it
        continue
    if not awardTitle:
        awardTitle = r['fundingReferences'][0]['awardTitle'] 
    if resourceType not in resourceType2NumOutputs:
        resourceType2NumOutputs[resourceType] = 0
    resourceType2NumOutputs[resourceType] += 1
    if not r['rights']:
        r['rights'].append({'rights': 'Not available'})
    for rights in r['rights']:
        right = xstr1(rights['rights'])
        if right not in right2NumOutputs:
            right2NumOutputs[right] = 0
        right2NumOutputs[right] += 1;
    
# Sort resource types by count of work desc
sorted_num_outputs = sorted(right2NumOutputs.items(),key=operator.itemgetter(1),reverse=True)
# Populate lists needed for pie chart
resource_types = [s[0] for s in sorted_num_outputs] 
num_outputs = [s[1] for s in sorted_num_outputs] 

# Generate a pie chart of number of grant outputs by resource type
fig = plt.figure()
ax = fig.add_axes([0,0,1,1])
ax.set_title('Number of Grant Outputs per License Type')
ax.axis('equal')
ax.pie(num_outputs, labels = resource_types,autopct='%1.2f%%')
plt.show()

Plot license type counts per resources type

Plot stacked bar plot showing how each type of FREYA grant award outputs contributes their license type counts to the corresponding aggregated total.

In [41]:
import plotly.io as pio
import plotly.express as px
from IPython.display import IFrame
import pandas as pd
from operator import itemgetter

xstr = lambda s: 'General' if s is None else str(s)
xstr1 = lambda s: 'Not available' if s is None else str(s)

# Adapted from: https://stackoverflow.com/questions/58766305/is-there-any-way-to-implement-stacked-or-grouped-bar-charts-in-plotly-express
def px_stacked_bar(df, color_name='License Type', y_name='Metrics', **pxargs):
    idx_col = df.index.name
    m = pd.melt(df.reset_index(), id_vars=idx_col, var_name=color_name, value_name=y_name)
    # For Plotly colour sequences see: https://plotly.com/python/discrete-color/     
    return px.bar(m, x=idx_col, y=y_name, color=color_name, **pxargs, 
                  color_discrete_sequence=px.colors.qualitative.Pastel1)
 
# Collect metric counts
funder = data['funder']
funderName = funder['name']

# Initialise dicts for the stacked bar plot
labels = {0: 'All Output Types'}

# Map each license type to a dict in turn mapping the position of the output's bar in plot 
# to the count of outputs corresponding to that license type
licenseType2Pos2Count = {}

# Collect output type labels and assign bar positions in plot
outputType2Pos = {}
pos = 1
for outputType, num in sorted(resourceType2NumOutputs.items(), key = itemgetter(1), reverse = True):
    labels[pos] = outputType
    outputType2Pos[outputType] = pos
    pos += 1
    
# Populate license type counts per output type (key = i) and add them to the aggregated counts (key: 0)
for node in funder['works']['nodes']:
    if node['versionOfCount'] > 0:
        # If the current output is a version of another one, exclude it
        continue       
    resource_type = xstr(node['types']['resourceType'])
    pos = outputType2Pos[resource_type]
    if len(node['rights']) == 0:
        node['rights'].append({'rights': 'Not available'})
    for rights in node['rights']:
        right = xstr1(rights['rights'])
        if right not in licenseType2Pos2Count:
            licenseType2Pos2Count[right] = {}
            for pos1, _ in enumerate(labels):
                # Initialise right's counts for each output type
                licenseType2Pos2Count[right][pos1] = 0
        licenseType2Pos2Count[right][0] += 1 
        licenseType2Pos2Count[right][pos] += 1
        
# Create stacked bar plot
x_name = "Output Types (sorted by number of outputs)"
dfDict = {x_name: labels}

for right in licenseType2Pos2Count:
    dfDict[right] = licenseType2Pos2Count[right]

df = pd.DataFrame(dfDict)
fig = px_stacked_bar(df.set_index(x_name), y_name = "Output Counts")

# Set plot background to transparent
fig.update_layout({
'plot_bgcolor': 'rgba(0, 0, 0, 0)',
'paper_bgcolor': 'rgba(0, 0, 0, 0)'
})

# Write interactive plot out to html file
pio.write_html(fig, file='out1.html')

# Display plot from the saved html file
display(Markdown("License Types of %s's [FREYA grant award](https://cordis.europa.eu/project/id/777523) outputs, shown per output type as a stacked bar plot:" % funderName))
IFrame(src="./out1.html", width=500, height=500)

License Types of European Commission's FREYA grant award outputs, shown per output type as a stacked bar plot:

Out[41]:
In [ ]: