Welcome to ECEN 760 in Fall 2017

Picture taken from Loeliger, "An Introduction to Factor graphs", IEEE Signal Processing Magazine, ™©® IEEE

Probablistic graphical models have emerged as a powerful language to represent interactions between several variables that arise in many engineering problems. The main ideas behind graphical models were developed somewhat independently in three different areas - statistical physics, artificial intelligence and coding theory. However, research over the past 20 years has resulted in a unified understanding of the main principles behind graphical models and has provided insight into how graphical models can be used to solve inference problems. It has been recognized that graphical models provide a framework for understanding several important algorithms used in diverse areas such as computer science, constraint satisfaction, computer vision, image processing, signal processing and coding theory. Graphical models also provide an important framework for various machine learning tasks and this is an active topic of research.

ECEN 760 is a graduate-level course that is roughly organized in three parts

  • Part I: Representing multi-variate distributions using graphical models such as Bayesian networks, Markov networks and factor graphs. Fundamental properties of these representations will be discussed and several examples from engineering will be presented.
  • Part II: Inference using graphical models - When is the problem of inferring the maximum likelihood or maximum a posteriori configuration of variables computationally easy? What is the relevance between computational efficiency and the underlying graphical structure?
  • Par III: Learning graphical models - How can we learn the structure of graphical models and their parameters from data (full or partial observations of the underlying variables)?

A tentative outline of the topics is given below. During Fall 2017, each lecture period will be 50 mins
Introduction 1
Review of probability, basic graph terminology 1
Representation using Bayesian networks, Markov networks, factor graphs 6
Examples from engineering and their graphical representation 4
Exact and approximate inference, Belief propagation, message passing 3
Generalized distributive law 2
Analysis of message passing algorithms on tree-like neighborhoods 3
Marginalization on trellises 2
Inference as optimization, Bethe free energy minimization 2
Linear programming decoder 2
Sampling, Monte Carlo techniques 4
Learning graphical models - Structure learning, parameter learning 6
Project presentations, midterm 4

  • Instructor : Krishna R. Narayanan (ude.umat|nrk#ude.umat|nrk)
  • Lecture time: MWF 11.30 - 12.20
  • Office Hours: T, Th 10.30 to 12.00 and by appointment otherwise at 334K Wisenbaker
  • Email : ude.umat|nrk#ude.umat|nrk
  • Recommended Reading :
    • D. Koller and N. Friedman, “Probabilistic Graphical Models : Principles and Techniques”, MIT Press
    • Lecture notes by Prof. Devavrat Shah, MIT
    • C. M. Bishop, "Pattern Recognition and Machine Learning", Springer
    • D. J. C. MacKay, "Information Theory, Inference and Learning", Cambridge University Press
    • M. Mezard and Montanari, “Information, Physics and Computation”, Oxford University Press
    • Research papers handed during the course
  • Prerequisites
    • Graduate level understanding of probability (ECEN 646 or equivalent)
    • Programming in a high-level language
    • Exposure to basic concepts in optimization will help. It can be picked up during the course.
    • A willingness to learn
  • Grading policy
    • Homeworks and Projects - 50%
    • Midterm - 25%
    • Final/Project - 25%

Americans with Disabilities Act (ADA) Policy Statement

The Americans with Disabilities Act (ADA) is a federal anti-discrimination statute that provides comprehensive civil rights protection for persons with disabilities. Among other things, this legislation requires that all students with disabilities be guaranteed a learning environment that provides for reasonable accommodation of their disabilities. If you believe you have a disability requiring an accommodation, please contact the Department of Student Life, Services for Students with Disabilities, in Cain Hall or call 845-1637.

Academic Integrity Statements

“An Aggie does not lie, cheat, or steal or tolerate those who do.”
Upon accepting admission to Texas A&M University, a student immediately assumes a commitment to uphold the Honor Code, to accept responsibility for learning, and to follow the philosophy and rules of the Honor System. Students will be required to state their commitment on examinations, research papers, and other academic work. Ignorance of the rules does not exclude any member of the TAMU community from the requirements or the processes of the Honor System.

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License