5SSD0 Course Syllabus¶

Learning Goals¶

• This course provides an introduction to Bayesian machine learning and information processing systems. The Bayesian approach affords a unified and consistent treatment of many useful information processing systems.
• Upon successful completion of the course, students should be able to:
  • understand the essence of the Bayesian approach to information processing.
  • specify a solution to an information processing problem as a Bayesian inference task on a probabilistic model.
  • design a probabilistic model by a specifying a likelihood function and prior distribution;
  • Code the solution in a probabilistic programming package.
  • execute the Bayesian inference task either analytically or approximately.
  • evaluate the resulting solution by examination of Bayesian evidence.
  • be aware of the properties of commonly used probability distribitions such as the Gaussian, Gamma and multinomial distribution; models such as hidden Markov models and Gaussian mixture models; and inference methods such as the Laplace approximation, variational Bayes and message passing in a factor graph.

Entrance Requirements (pre-knowledge)¶

• Undergraduate courses in Linear Algebra and Probability Theory (or Statistics).
• Some scientific programming experience, eg in MATLAB or Python. In this class, we use the Julia programming language, which has a similar syntax to MATLAB, but is (close to) as fast as C.

Important Links¶

• Please bookmark the following three websites:

  1. The course homepage http://bmlip.nl (or try https://biaslab.github.io/teaching/bmlip ) contains links to all materials such as lecture notes and video lectures.
  2. The Piazza course site will be used for Q&A and communication.
  3. The Canvas course site will be sparingly used for communication (mostly by ESA staff)

Materials¶

• All materials can be accessed from the course homepage.

• Materials consist of the following resources:

  • Lecture notes (mandatory)
  • Probabilistic Programming (PP) notes (mandatory)
  • optional materials to help understand the lecture and PP notes
    • exercises
    • Q&A at Piazza
    • practice exams

• In previous years, we also posted a Video Guide for each lecture. These videos were created during the Corona epidemic (2020) and aim to cover the main points or (expected) sticky issues in a lecture. They are getting a bit outdated but are still available via the 5SSD0 site in the 2022/23 year. Of course, these videos are very optional but may help you focus on the main points of the class.

• Source materials are available at github repo at https://github.com/bertdv/BMLIP. You do not need to bother with this site. If you spot an error in the materials, please raise the issue at Piazza.

Study Guide¶

• The lecture notes and probabilistic programming notes contain the mandatory materials. Some lecture notes are extended by a reading assignment, see the first cell in the lecture notes. These reading assignment are also part of the mandatory materials.

• Slides that are not required for the exam are moved to the end of the notes and preceded by an OPTIONAL SLIDES header.

• Please study the lecture notes before you come to class!!

• Any sticky issues regarding the lecture notes?

  • Pose you question at the Piazza site!
  • Your questions will be answered at the Piazza site by fellow students and accorded (or corrected) by the teaching staff.

• Then come to class

  • During the scheduled classroom meetings, I will not teach all materials in the lecture notes.
  • Rather, I will first discuss a summary of the lecture notes and then be available for any additional questions that you may still have.

• Each class also comes with a set of exercises. They are often a bit challenging and test more of your quantitative skills than you will need for the exam. When doing exercises, feel free to make use of Sam Roweis' cheat sheets for Matrix identities and Gaussian identities. Also accessible from course homepage.

Piazza (Q&A)¶

• We will be using Piazza for Q&A and news. The system is highly catered to getting you help fast and efficiently from both classmates and the teaching staff.

• Sign up for Piazza today if you have not done so. And install the Piazza app on your phone!

• The quicker you begin asking questions on Piazza (rather than via emails), the quicker you'll benefit from the collective knowledge of your classmates and instructors. We encourage you to ask questions when you're struggling to understand a concept—you can even do so anonymously.

• We will also disseminate news and announcements via Piazza.

• Unless it is a personal issue, pose your course-related questions at Piazza (in the right folder).

• Please contribute to the class by answering questions at Piazza.

  • If so desired, you can contribute anonymously.
  • Answering technical questions at Piazza is a great way to learn. If you really want to understand a topic, you should try to explain it to others.
  • Every question has just a single students' answer that students can edit collectively (and a single instructors’ answer for instructors).

• You can use LaTeX in Piazza for math (and please do so!).

• Piazza has a great search feature. Use search before putting in new questions.

Exam Guide¶

• There will be a written exam in multiple-choice format.

• You are not allowed to use books nor bring printed or handwritten formula sheets to the exam. Difficult-to-remember formulas are supplied at the exam sheet.

• No smartphones at the exam.

• The tested material consists of the mandatory lecture + PP notes (+ mandatory reading assignments as assigned in the first cell/slide of each lecture notebook).

• The class homepage contains two representative practice exams from previous terms.

OPTIONAL SLIDES ¶

Title¶

• The slides below the OPTIONAL SLIDES marker are optional for the exam.