#!/usr/bin/env python
# coding: utf-8

# <script async src="https://www.googletagmanager.com/gtag/js?id=UA-59152712-8"></script>
# <script>
#   window.dataLayer = window.dataLayer || [];
#   function gtag(){dataLayer.push(arguments);}
#   gtag('js', new Date());
# 
#   gtag('config', 'UA-59152712-8');
# </script>
# 
# # BlackHoles@Home Tutorial: Compiling the `BOINC` server on Linux
# 
# ## Author: Leo Werneck
# 
# ## This tutorial notebook demonstrates how to compile the `BOINC` server Linux. It focuses specifically on [Ubuntu](https://ubuntu.com), but adapting the scripts for other Linux flavors should be straightforward
# 
# ## Introduction:
# 
# The [BlackHoles@Home](http://blackholesathome.net/) project allows users to volunteer CPU time so a large number of binary black holes simulations can be performed. The objective is to create a large catalog of [gravitational waveforms](https://en.wikipedia.org/wiki/Gravitational_wave), which can be used by observatories such as [LIGO](https://www.ligo.org), [VIRGO](https://www.virgo-gw.eu), and, in the future, [LISA](https://lisa.nasa.gov) in order to infer what was the source of a detected gravitational wave.
# 
# BlackHoles@Home is destined to run on the [BOINC](https://boinc.berkeley.edu) infrastructure (alongside [Einstein@Home](https://einsteinathome.org/) and [many other great projects](https://boinc.berkeley.edu/projects.php)), enabling anyone with a computer to contribute to the construction of the largest numerical relativity gravitational wave catalogs ever produced.
# 
# ### Additional Reading Material:
# 
# * [BOINC's Wiki page](https://boinc.berkeley.edu/trac/wiki)
# * [Debian's Wiki BOINC server guide](https://wiki.debian.org/BOINC/ServerGuide/Initialisation)

# <a id='toc'></a>
# 
# # Table of Contents
# $$\label{toc}$$
# 
# This tutorial compiles the `BOINC` server. It will also install all needed dependencies. We also provide a script to set up the server correctly.
# 
# 1. [Step 1](#loading_python_nrpy_modules): Loading necessary Python/NRPy+ modules
# 1. [Step 2](#compilation_script): A simple script to compile the `BOINC` server    
# 1. [Step 3](#server_setup): Setting up the server
# 1. [Step 4](#latex_pdf_output): Output this notebook to $\LaTeX$-formatted PDF file

# <a id='loading_python_nrpy_modules'></a>
# 
# # Step 1: Loading needed Python/NRPy+ modules \[Back to [top](#toc)\]
# $$\label{loading_python_nrpy_modules}$$
# 
# We start by loading the necessary Python/NRPy+ moduels used by this tutorial notebook. We also set up the `BOINC` directory path (the default path is the current working directory).

# In[1]:


# Step 1: Import necessary modules - set directories.
# Step 1.a: Import required Python modules
import sys

# Step 1.b: Add NRPy's root directory to the sys.path()
sys.path.append("..")

# Step 1.c: Load NRPy+'s command line helper module
import cmdline_helper as cmd # NRPy+: Multi-platform Python command-line interface


# <a id='compilation_script'></a>
# 
# # Step 2: A simple script to compile the `BOINC` server \[Back to [top](#toc)\]
# $$\label{compilation_script}$$
# 
# We will now write a simple `bash` script that will take care of downloading the `BOINC` source code and compiling the `BOINC` server.
# 
# The script performs the following tasks:
# 
# 1. Install the necessary dependencies.
# 1. Download the [`BOINC` source code](https://github.com/BOINC/boinc) (if necessary).
# 1. Compile the `BOINC` server and libraries.
# 
# The following packages are required for a successfull compilation:
# 
# 1. git
# 1. make
# 1. m4
# 1. autoconf
# 1. libtool
# 1. A C++ compiler
# 1. Python
# 1. pip
# 1. Python-MySQL (installed using pip)
# 1. mysql-server
# 1. apache2
# 1. php (with cli, gd, and mysql support)
# 1. pkg-config
# 1. Development version of libmysql++
# 1. Development version of libssl
# 1. Development version of libcurl with openssl support

# In[2]:


get_ipython().run_cell_magic('writefile', 'compile_BOINC_server.sh', '#!/bin/bash\n# Install all required software\nsudo apt-get install -y \\\n  git make m4 autoconf libtool g++ python3 python-is-python3 python3-pip \\\n  mysql-server apache2 php php-cli php-gd php-mysql pkg-config \\\n  libmysql++-dev libssl-dev libcurl4-openssl-dev\n\n# Now use pip to install MySQL for python\npip3 install mysql-client\n\n# Download BOINC repository (if necessary)\nboincdir="$(pwd)/boinc"\nif [ -d "$boincdir" ]; then\n  echo "BOINC directory found at $boincdir"\nelse\n  echo "BOINC directory not found at $boincdir. Cloning from BOINC github repository..."\n  git clone https://github.com/BOINC/boinc boinc\nfi\n\n# Now change directories to the BOINC directory\ncd "$boincdir"\n\n# Compile the core BOINC server and libraries\n./_autosetup -f\n./configure --disable-client --disable-manager\nmake\n\n# Make sure the boinc_zip library is compiled\ncd "${boincdir}/zip"\nmake\n')


# <a id='server_setup'></a>
# 
# # Step 3: Setting up the server \[Back to [top](#toc)\]
# $$\label{server_setup}$$
# 
# We now generate a bash script which will set up the server for us. After running the `BOINC` executables that set up a new server, some file/directory permissions are still missing/misconfigured, and therefore some fixes are necessary. The script below takes care of all of the problems we have encountered when setting up a `BOINC` server so far.
# 
# The script will also add your apps to the server, if you have any. Simply change tthe `addonsdir` variable in the code below to point to the directory of your apps base directory. It assumes that the `addonsdir` directory contains at least the folders `apps` and `templates`. The `apps` directory should contain your `BOINC` applications, which should follow the `BOINC` standards (see e.g. the directory tree in [this `BOINC` tutorial](https://boinc.berkeley.edu/trac/wiki/WrapperApp)), as well as a file called `app_list.txt`.
# 
# The `app_list.txt` file should have two columns: the first one containing thte application name and the second one containing the application name in user friendly format. Lines starting with "#" are treated as comments. Here is a brief example:
# 
# ```bash
# # Lines starting with a # are treated as comments
# 
# # Empty lines are also OK
# 
# # The file should contain 2 columns: the first one
# # with the application's (short) name and the second
# # with the application's user-friendly name. Here's
# # an example:
# 
# # My BOINC applications
# 
# my_app1 MyApplication1
# my_app2 MyApplication2
# my_app3 MyFancyApp3
# ```
# 
# <font color=red>NOTE:</font> the script below will request your MySQL root and your root passwords. Because of this, we will not be running it in this tutorial notebook, but you can do it by first adjusting the first few variables in the script (which set up the project and database name, as well as installation directories) and then running:
# 
# ```bash
# $: source setup_BOINC_server.sh
# ```
# 
# Once the script below finishes running without errors, you can go into the project's root directory and execute
# 
# ```bash
# $: sudo ./bin/start
# ```
# 
# to start the server. If everything went according to plan, you should then be able to access the project's website using the URL
# 
# ```
# http://<myip>/<projectname>
# ```
# 
# where `<projectname>` should be replaced with the project's name and `<myip>` by the server's IP address. If you haven't modified this on the script below, then the value of `<myip>` can be found by running
# 
# ```bash
# $: hostname -I | awk '{print $1}'
# ```
# 
# To stop the server, go to the project's root directory and execute
# 
# ```bash
# $: sudo ./bin/stop
# ```
# 
# will stop the server.

# In[3]:


get_ipython().run_cell_magic('writefile', 'setup_BOINC_server.sh', '#!/bin/bash\n\n# MySQL variable: project\'s database name.\ndbname=blackholesathome\n\n# MySQL variable: project\'s database user (I\'m user my linux user here).\ndbuser=bhahadm\n\n# MySQL variable: project\'s database password.\ndbpasswd=P@ssword1234\n\n# BOINC variable: project name (for e.g. directories and files).\nprojectname=blackholesathome\n\n# BOINC variable: project nice name (for displaying on e.g. the website).\nprojectnicename="BlackHoles@Home"\n\n# BOINC variable: directory in which the project will be installed.\ninstallroot="$(pwd)/projects"\n\n# BOINC variable: BOINC source code directory.\nboincroot="$(pwd)/boinc"\n\n# BOINC variable: project directory.\nprojectroot="${installroot}/${projectname}"\n\n# BOINC variable: this variable will be used to set your project\'s\n#                 webpage ID. If you are running a production\n#                 server, then you might want to add your static.\n#                 IP address here\nmyip=$(hostname -I | awk \'{print $1}\')\n\n# BOINC variable: project\'s webpage address.\nhosturl="http://${myip}"\n\n# Set this to a directory containing your BOINC applications.\n# The script expects "apps" and "templates" to be subdirectories.\n# If empty, then it will be unused.\naddonsdir=""\nappsdir=$addonsdir/apps\n\n# Here we will pipe commands to the MySQL shell. The commands\n# below set up a new database user, whose handle is specified\n# by the dbuser variable set above. We then set up a database\n# and grant permission to the user so that we are able to\n# modify the database as we please.\nprintf "Please enter your MySQL password.\\n"\ncat <<EOMYSQL | mysql -u root -p;\nDROP USER \'$dbuser\'@\'localhost\';\nDROP DATABASE IF EXISTS $dbname;\nCREATE USER \'$dbuser\'@\'localhost\' IDENTIFIED BY \'$dbpasswd\';\nGRANT ALL PRIVILEGES ON $dbname.* TO \'$dbuser\'@\'localhost\';\nEOMYSQL\n\n# Now we create our project by running BOINC\'s make_project\n# script.\n# NOTE: if you are getting errors with "MySQL" module not\n#       found in Python, then you might need to install\n#       the MySQL Python client using sudo, i.e.:\n#\n#       sudo pip install mysql-client\nprintf "Please enter your sudo password.\\n"\nsudo mkdir -p "$installroot"\nsudo "${boincroot}/tools/make_project" \\\n     --srcdir="$boincroot" \\\n     --url_base "$hosturl" \\\n     --db_name "$dbname" \\\n     --db_user "$dbuser" \\\n     --db_passwd "$dbpasswd" \\\n     --delete_prev_inst \\\n     --drop_db_first \\\n     --project_root "$projectroot" \\\n     "$projectname" "$projectnicename"\n\n# Fix permissions of project directories. If this\n# is not done, then the website is usually not\n# displayed correctly. On top of that, users often\n# have difficulty communicating with the server.\ncd "$installroot/$projectname"\nsudo chown bhahadm:bhahadm -R .\nsudo chmod g+w -R .\nsudo chmod 755 -R upload html/cache html/inc html/languages html/languages/compiled html/user_profile\nhostname=`hostname`\nsudo chgrp -R www-data log_"$hostname" upload\n\n# Further permission fixes\necho -n "html/inc: "; sudo chmod o+x html/inc && sudo chmod -R o+r html/inc && echo "[ok]" || echo "[failed]"\necho -n "html/languages: "; sudo chmod o+x html/languages/ html/languages/compiled && echo "[ok]" || echo "[failed]"\n\n# Copy webpage configuration to apache directory\nsudo cp "${projectroot}/${projectname}.httpd.conf" /etc/apache2/sites-available\ncd "${projectroot}" && sudo a2ensite "${projectname}.httpd.conf"\nsudo /etc/init.d/apache2 reload\n\n# Make sure cgi and php modules are loaded\nsudo a2enmod cgi\nsudo a2enmod php7.4\nsudo /etc/init.d/apache2 restart\n\n# Add the project name to the webpage file, replacing the default values.\nsed -i "s/REPLACE WITH PROJECT NAME/${projectnicename}/g" "${projectroot}/html/project/project.inc"\nsed -i "s/REPLACE WITH COPYRIGHT HOLDER/${projectnicename} Team/g" "${projectroot}/html/project/project.inc"\n\n# Copy user applications apps to the project\'s directory\nif [ "$appsdir" != "" ]; then\n\n    # Start by copying all the apps in the application directory\n    # to the project\'s apps directory. We also remove the "app_list.txt"\n    # file, which is not used by the BOINC project.\n    cp -r $appsdir/* "${projectroot}/apps/" && rm "${projectroot}/apps/app_list.txt"\n\n    # Now we add our apps to the project by creating entries in\n    # the project\'s project.xml file.\n    # First delete the last line of the file (we\'ll add it back later)\n    sed -i \'$ d\' "${projectroot}/project.xml"\n\n    # Then add the apps to the file\n    while read p; do\n\n        # Skip if line is a comment\n        [[ $p =~ ^#.* ]] && continue\n\n        stringarray=($p)\n        echo "\n    <app>\n        <name>${stringarray[0]}</name>\n        <user_friendly_name>${stringarray[1]}</user_friendly_name>\n    </app>"\n\n    done < "${appsdir}/app_list.txt" >> "${projectroot}/project.xml"\n\n    # Add the last line back\n    echo "</boinc>" >> "${projectroot}/project.xml"\n\n    # Copy templates to server\n    cp ${addonsdir}/templates/* ${projectroot}/templates/\n\n    # Add all the apps to the project\'s database\n    $projectroot/bin/xadd\n\n    # Create new app versions\n    $projectroot/bin/update_versions\n\nfi # if [ "$appsdir" != "" ]\n\n# All done!\nprintf "All done!\\n"\n')


# <a id='latex_pdf_output'></a>
# 
# # Step 4: Output this notebook to $\LaTeX$-formatted PDF file \[Back to [top](#toc)\]
# $$\label{latex_pdf_output}$$
# 
# The following code cell converts this Jupyter notebook into a proper, clickable $\LaTeX$-formatted PDF file. After the cell is successfully run, the generated PDF may be found in the root NRPy+ tutorial directory, with filename
# [Tutorial-BlackHolesAtHome-Compiling_the_BOINC_server_on_Linux.pdf](Tutorial-BlackHolesAtHome-Compiling_the_BOINC_server_on_Linux.pdf) (Note that clicking on this link may not work; you may need to open the PDF file through another means.)

# In[4]:


get_ipython().system('cp ../latex_nrpy_style.tplx .')
cmd.output_Jupyter_notebook_to_LaTeXed_PDF("Tutorial-BlackHolesAtHome-Compiling_the_BOINC_server_on_Linux")
get_ipython().system('rm -f latex_nrpy_style.tplx')