Foundations of Artificial Intelligence (FAI) Group

Seminar: Neural Networks in AI Planning (NNPLAN)

Presentation date: February 21 in MS Teams

The seminar will be run in a block format. There will be an initial meeting on Monday, October 25, 16:15-17:45. All student presentations will be given on a single day after the end of term. A detailed schedule is given below.

All meetings will take place in MS Teams . The seminar language is English throughout.

Supervisors for the seminar are Daniel Höller, Daniel Fišer, Marcel Steinmetz, Patrick Ferber, and Jörg Hoffmann

Your task will be to read and understand a piece of research, to write a summary paper in your own words, to give a presentation, and to provide detailed feedback for the paper and presentation of a fellow student.

All email interaction must be pre-fixed with "[NNPLAN21-22]" in the email subject.

No plagiarism. It is Ok (and encouraged!) to use web resources to further your understanding of your assigned topic. Especially, the video presentations of the authors are helpful starting points. However, it is inadmissible to use pieces of such material for your summary paper or presentation. Any plagiarism will result in disqualification from the seminar.

Content. Planning is the sub-area of AI concerned with complex action-choice problems, which occur in a broad range of applications ranging from game playing to smart production. Learning is a natural approach to planning effectively in a given application, and recent results on complex board games (AlphaGo/Zero systems series) has shown the power of this approach. Yet beyond board games this approach is still in its infancy, and strong generalization across structure such as different goals and scaling instance size remains a widely open research problem. The seminar covers works at the current research frontier investigating neural architectures in general planning.

Prerequisites. Participants must have successfully completed an Artificial Intelligence core course. They should be familiar with automatic planning at least to the extent of the material covered in the Artificial Intelligence course; successful participation in one of our AI Planning courses will be an advantage, but is not absolutely necessary to follow the seminar. Furthermore, good basic knowledge on neural networks is required. Ideally, at least one lecture relevant to neural networks was completed.

Registration. Is via the central seminar registration system.

Grading. The final grading will be based, in this order of importance, on:

• The quality of your final presentation.
• The quality of your final summary paper.
• The quality of the feedback you provide to your mentee student (see below).
• Your participation in the discussions during the block seminar.

Summary Paper. For the summary paper, you must use this tex template. You are required to read at least 2 related papers, for the related work section. You are allowed to modify the section structure given in the template if, for whatever reason, this is more adequate for the work you are summarizing.

The seminar paper should be about 4 pages long (not counting the literature list, and in the double-column format of the template). This is a rough guideline, not a strict rule. If you need, say, 5-6 pages to do your paper justice then definitely do so.

Schedule and Deadlines (tentative!).

• October 25, 16:15-17:45: Initial meeting.

• October 27: Send a ranked list of the topics you would like to take, by email to Daniel Höller. That is, send something like "area 1.2, area 2.2, area 1.1, area 4.2, area 3.1". Please include into this list all topics that you would be willing to accept. The list must contain at least 5 topics.

  Note that each topic is associated with a mentee student (to whom you will provide feedback, see the following deadlines); and a mentor student (who will provide feedback to you, see the following deadlines). The mentee/mentor assignment will be a "cycle" through each of the four topic areas as listed below: within areas with 2 topics, the two students mentor each other; within areas with 3 topics, the student with topic 1 mentors the student with topic 2, who mentors the student with topic 3, who mentors the student with topic 1. If you want to team up with someone specific, please do state that in your email.

• October 29: Receive your topic.

  Read the material associated with your topic carefully, and prepare an initial version of your summary paper, using the tex template given above.

• November 22-26: Make an appointment with your supervisor to discuss your paper. You should give a brief oral summary of the paper (5--10 minutes). Take the opportunity to ask questions.

  NOTE: The following deadlines marked with "(ca.)" are meant as a guideline. You are required to do these things, but if you do them 3-4 days earlier or later, that is no problem.

• December 8 (ca.): Send your summary paper to your mentor student (cc supervisors).
• December 12 (ca.): Send feedback regarding the summary paper to your mentee student (cc supervisors).
• December 19 (ca.): Send revised summary paper to your mentor student (cc supervisors).
• January 12 (ca.): Send feedback regarding the revised summary paper to your mentee student (cc supervisors).
• January 19 (ca.): Send presentation slides to mentor student (cc supervisors).
• January 26 (ca.): Send feedback regarding the presentation slides to your mentee student (cc supervisors).
• February 2 (ca.): Send revised presentation slides to mentor student (cc supervisors).
• February 9 (ca.): Send feedback regarding the revised presentation slides to your mentee student (cc supervisors).

• February 15: Send your final summary paper by email to your supervisor.
• February 16-23 (ca.): Give a presentation (20 minutes talk, plus 10 minutes discussion) in the block seminar. Attendance to all talks is required. Please try to stick to the 20 minutes time slot for your talk; it should not be a lot shorter, nor a lot longer.

Topics. Each participant will be assigned one topic, each of which consists of one paper. The overall amount and difficulty of the material associated with each topic is roughly balanced.

Area 1: Supervised learning of heuristics (supervisor: Jörg Hoffmann)

1. Ferber, Helmert, Hoffmann: Neural Network Heuristics for Classical Planning: A Study of Hyperparameter Space. ECAI 2020: 2346-2353.
   Paper: Area 1 Topic 1
2. Karia, Srivastava: Learning Generalized Relational Heuristic Networks for Model-Agnostic Planning. AAAI 2021.
   Paper: Area 1 Topic 2

1. Hamilton: Graph Representation Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning. 1-159. Chapter 5. Preprint (2021)
   Paper: Area 2 Topic 1
2. Shen, Trevizan, Thiébaux: Learning Domain-Independent Planning Heuristics with Hypergraph Networks. ICAPS 2020: 574-584.
   Paper: Area 2 Topic 2

Area 3: Reinforcement learning of heuristic functions (supervisor: Marcel Steinmetz)

1. Arfaee, Zilles, Holte: Bootstrap Learning of Heuristic Functions. SOCS 2010.
   Paper: Area 3 Topic 1
   
2. Ferber, Hoffmann, Helmert: Neural Network Heuristic Functions for Classical Planning: Reinforcement Learning and Comparison to Other Methods. PRL @ ICAPS 2021.
   Paper: Area 3 Topic 2

Area 4: Learning action policies (supervisor: Daniel Höller)

1. Toyer, Trevizan, Thiébaux, Xie: Action Schema Networks: Generalised Policies with Deep Learning. AAAI 2018: 6294-6301.
   Paper: Area 4 Topic 1
2. Groshev, Goldstein, Tamar, Srivastava, Abbeel: Learning Generalized Reactive Policies Using Deep Neural Networks. ICAPS 2018: 408-416.
   Paper: Area 4 Topic 2

Area 5: Dynamic Algorithm Configuration (supervisor: Daniel Fišer)

1. Gomoluch, Alrajeh, Russo, Bucchiarone: Learning Neural Search Policies for Classical Planning. ICAPS 2020: 522-530.
   Paper: Area 5 Topic 1
2. Speck, Biedenkapp, Hutter, Mattmüller, Lindauer: Learning Heuristic Selection with Dynamic Algorithm Configuration. ICAPS 2021.
   Paper: Area 5 Topic 2