#!/usr/bin/env python
# coding: utf-8

# Graphical Lasso
# =======================
# 
# *Important:* Please read the [installation page](http://gpeyre.github.io/numerical-tours/installation_python/) for details about how to install the toolboxes.
# $\newcommand{\dotp}[2]{\langle #1, #2 \rangle}$
# $\newcommand{\enscond}[2]{\lbrace #1, #2 \rbrace}$
# $\newcommand{\pd}[2]{ \frac{ \partial #1}{\partial #2} }$
# $\newcommand{\umin}[1]{\underset{#1}{\min}\;}$
# $\newcommand{\umax}[1]{\underset{#1}{\max}\;}$
# $\newcommand{\umin}[1]{\underset{#1}{\min}\;}$
# $\newcommand{\uargmin}[1]{\underset{#1}{argmin}\;}$
# $\newcommand{\norm}[1]{\|#1\|}$
# $\newcommand{\abs}[1]{\left|#1\right|}$
# $\newcommand{\choice}[1]{ \left\{  \begin{array}{l} #1 \end{array} \right. }$
# $\newcommand{\pa}[1]{\left(#1\right)}$
# $\newcommand{\diag}[1]{{diag}\left( #1 \right)}$
# $\newcommand{\qandq}{\quad\text{and}\quad}$
# $\newcommand{\qwhereq}{\quad\text{where}\quad}$
# $\newcommand{\qifq}{ \quad \text{if} \quad }$
# $\newcommand{\qarrq}{ \quad \Longrightarrow \quad }$
# $\newcommand{\ZZ}{\mathbb{Z}}$
# $\newcommand{\CC}{\mathbb{C}}$
# $\newcommand{\RR}{\mathbb{R}}$
# $\newcommand{\EE}{\mathbb{E}}$
# $\newcommand{\Zz}{\mathcal{Z}}$
# $\newcommand{\Ww}{\mathcal{W}}$
# $\newcommand{\Vv}{\mathcal{V}}$
# $\newcommand{\Nn}{\mathcal{N}}$
# $\newcommand{\NN}{\mathcal{N}}$
# $\newcommand{\Hh}{\mathcal{H}}$
# $\newcommand{\Bb}{\mathcal{B}}$
# $\newcommand{\Ee}{\mathcal{E}}$
# $\newcommand{\Cc}{\mathcal{C}}$
# $\newcommand{\Gg}{\mathcal{G}}$
# $\newcommand{\Ss}{\mathcal{S}}$
# $\newcommand{\Pp}{\mathcal{P}}$
# $\newcommand{\Ff}{\mathcal{F}}$
# $\newcommand{\Xx}{\mathcal{X}}$
# $\newcommand{\Mm}{\mathcal{M}}$
# $\newcommand{\Ii}{\mathcal{I}}$
# $\newcommand{\Dd}{\mathcal{D}}$
# $\newcommand{\Ll}{\mathcal{L}}$
# $\newcommand{\Tt}{\mathcal{T}}$
# $\newcommand{\si}{\sigma}$
# $\newcommand{\al}{\alpha}$
# $\newcommand{\la}{\lambda}$
# $\newcommand{\ga}{\gamma}$
# $\newcommand{\Ga}{\Gamma}$
# $\newcommand{\La}{\Lambda}$
# $\newcommand{\si}{\sigma}$
# $\newcommand{\Si}{\Sigma}$
# $\newcommand{\be}{\beta}$
# $\newcommand{\de}{\delta}$
# $\newcommand{\De}{\Delta}$
# $\newcommand{\phi}{\varphi}$
# $\newcommand{\th}{\theta}$
# $\newcommand{\om}{\omega}$
# $\newcommand{\Om}{\Omega}$
# $\newcommand{\eqdef}{\equiv}$

# This tour details the graphical Lasso method for covariance estimation. The idea of using sparsity of the precision matrix to regularize covariance estimation appears in several works, including [MeinshausenBuhlmann06]. The use of an $\ell^1$ penalty, together with the block-coordinate optimization scheme presented here, is introduced in  [AspremontBanerjeeElGhaoui08,BanerjeeElGhaouiAspremont08]. This algorithm was popularized by [FriedmanHastieTibshirani08] under the name "Graphical Lasso". 

# In[1]:


import numpy as np
import matplotlib.pyplot as plt


# Bibliography
# ==============
# 
# - [MeinshausenBuhlmann06] N. Meinshausen and P. Buhlmann. [High dimensional graphs and variable selection with the lasso](https://projecteuclid.org/euclid.aos/1152540754). Annals of statistics, 34:1436–1462, 2006.
# - [AspremontBanerjeeElGhaoui08]  Alexandre d'Aspremont, Onureena Banerjee, Laurent El Ghaoui, [First-order methods for sparse covariance selection](https://arxiv.org/abs/math/0609812), SIAM J. Matrix Anal. Appl., 30(1), 56–66, 2008.
# - [BanerjeeElGhaouiAspremont08] Onureena Banerjee, Laurent El Ghaoui, Alexandre d'Aspremont, [Model Selection Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data](https://www.jmlr.org/papers/v9/banerjee08a.html), JMLR 9(15):485−516, 2008.
# - [FriedmanHastieTibshirani08] Sparse inverse covariance estimation with the graphical lasso, Jerome Friedman, Trevor Hastie, Robert Tibshirani, Biostatistics, 9(3):432-41, 2008.

# In[ ]: