The term course project is an opportunity for students to create new components for Shutter, the robot photographer. Projects will generally take the whole semester to complete, from conception to literature review, implementation and evaluation. In general, projects are expected to be several thousand lines of code and implemented at a high level of proficiency in Python. ROS tools should be used for integration with Shutter.

Overview

Potential projects would generally fall into two categories:

  • Applications: The robot is enabled with a new capability that can be deployed and experienced by users. System integration is one of the main contributions for this type of work.

  • Algorithms: A new algorithm (or model), or a variant of an existing algorithm, is built to tackle a specific perception or decision making task. While system integration is relevant, the main contribution is an algorithm or model.

The main restrictions for projects are that:

  1. The project is relevant to a situated agent like Shutter. While projects may use simulations extensively for the bulk of the work, they should all demonstrate applicability (e.g., through a simple demo) on a real or simulated Shutter robot.

  2. It is allowed for students to work with additional hardware that they own or have access to through research labs. The Yale Interactive Machines Group may be able to provide additional cameras, microphones, or other hardware as well upon request – interested students should contact the instructor for more information.

Students are encouraged to work on research problems for their final projects. Projects that aim to replicate results from existing papers are allowed as long as they have a clear evaluation plan, and a controlled experiments are conducted to systematically evaluate the results.

Timeline of Important Events

Tue. 9/13 - Brainstorm term project ideas in class
Fri. 9/16 - Complete online form with team information for the term project
Thu. 9/22 - Term project proposal is due (graded deliverable)
Tue. 10/25 - Term project feedback from peers
Tue. 11/08 - Term project milestone report is due (graded deliverable)
Thu. 12/01 - Demo Day: project previews (graded deliverable)
Thu. 12/08 - Term project presentations (graded deliverable)
Thu. 12/15 - Term project reports, videos & supplementary are due (graded deliverable)

Project Ideas

An initial set of project ideas can be seen in this document. For questions about these ideas, email the corresponding contact person.

Deliverables

The projects will have 5 main deliverables:

  1. Project Proposal (8/50 points; due 9/22/22): The first deliverable is a project proposal (no longer than 4 pages excluding references and images) that describes:

    • the main goals for the project, i.e., what problem will the project focus on or help solve?;
    • the motivation for the project, i.e., why is the problem interesting or relevant?;
    • the approach to solve the problem of interest (including a detailed description of how the proposed software system will be organized, e.g., what ROS nodes do you need to implement vs. which existing nodes will you leverage for your project? what information will these nodes transfer among each other?);
    • how will results be evaluated;
    • a description of any hardware needed to complete the project (may it be provided by course instructors or available to students elsewhere);
    • a working plan to accomplish the set goals, including what is expected to be accomplished by the milestone deadline and by the final project deadline; and
    • a plan for distributing the work among the students involved in the project, i.e., who will work on what?.

    If the project will use of machine learning, then the project proposal must specify what dataset will be used to train and evaluate models. Due to the limited time to complete the project, we discourage big data collection efforts and, rather, suggest students leverage pre-existing datasets for algorithmic projects (unless the project is about learning from humans or online learning).

    For application projects, students can use pre-trained models if need be. Note that in the latter case, it is important that students explain well the challenges that building their system would entail. That is, it should be clear from the proposal that there are interesting/hard problems to be solved by the students through their system implementaion (e.g., challenges may be related to achieving low execution time or increased robustness to diverse testing conditions).

  2. Project Milestone (13/50 points; due 11/08/22): The second deliverable is a report with intermediary results. The report should be between 3-4 pages (pdf format) and should be formatted according to the 2017 ACM Master Article Template. Note that the ACM LaTeX template is available through Overleaf here. Please use the template in the two column format: comment out the “manuscript” document class if it’s enabled, and uncomment the “sigconf” document class as in this image.

    The milestone report should include:

    • Title and a list of authors;
    • Introduction: motivation, explanation of the problem of interest, and description of the overall (high-level) plan for approaching the problem;
    • Related Work: list of prior work that helps contextualize the effort;
    • Technical Approach: description of the system or methods that you are applying to solve the problem of interest;
    • Intermediate/Preliminary Results: a statement of the results and any evaluation up to the milestone deadline.
    • Summary of Contributions: Statement by each of the group members of the effort and results that they have contributed to the project.
    • Updated Working Plan: New working plan taking into the consideration the progress made so far and challenges (should include distribution of work among students given updated plan).

    Along with the report, students should submit a zip file including the current project source code and any trained models (e.g., checkpoints for neural networks) if you used machine learning for your project.

  3. Project Demo (5/50 points; due 12/01/22): In-person, short project previews. Students demonstrate tangible progress towards their project goals and integration of technologies in Shutter. Course staff and students test out existing implementation.

  4. Project Presentation (10/50 points; due 12/08/22): 10 min project presentation highlighting the main goals of the project, approach, and results. Students are encouraged to show short videos about parts of their project or the whole system that was developed.

  5. Final Report and Supplementary Material (14/50 points; due 12/15/22): Final project write-up (and accompanying video) and supplementary material (source code).

    The report should be between 6-8 pages and be formatted according to the 2017 ACM Master Article Template. Please use the template in the two column format: comment out the “manuscript” document class if it’s enabled, and uncomment the “sigconf” document class as in this image.

    The report should be structured like a paper from a conference (e.g., CHI, HRI, SIGGRAPH, CVPR, etc.). In particular, we suggest the following structure for the final report:

    • Title and a list of authors;
    • Abstract: brief description of the problem, approach, and key results. Should be no more than 300 words.
    • Introduction: explanation of the problem of interest, and overall plan for approaching the problem;
    • Related Work: list of prior work that helps contextualize the effort;
    • Method: description of the approach for solving the problem of interest;
    • Experiments: description of the experiments that were performed to test the proposed system or evaluate its components;
    • Conclusion and Future Work: summary of key results and a discussion of potential future work.

    Students must create a brief video that documents how their system works and how people interacted with it if appropriate. The video should be created in an mp4 container with H.264 format, which is portable and easily playable. The video submission should not exceed 50MB in size. Tip: HandBrake can be used for free to encode video.

    Students must also provide the following supplementary material as part of their final project submission:

    • A zip file including relevant materials that are not available online, including the project source code and any trained models (e.g., checkpoints for neural networks) if you used machine learning for your project.
    • A document describing how to run the code and any other relevant supplementary material to help understand the progress that you made towards your goals (there is no page limit for this document). This document must include a statement of contributions for each of the members of the team. Each statement must summarize the contributions that the corresponding student made to the project from the beginning of the collaboration. The document may also include a link to a website documenting results from experiments, interactive visualizations of the perception part of your system, etc.

☝ There are no late days for any of the final project deliverables.

Collaboration Policy

Students should work in small teams of 2 or 3 people.

Note that it is expected that each student will program 1200 lines of code for the semester-long project if not more. This code will likely be a combination of algorithms and systems code. Because each student should be programming significantly for the semester-long project, groups with 3 students are expected to complete bigger/more ambitiious projects during the semester than groups with 2 students.

Honor Code

It is expected that students will use libraries for scientific computing (e.g., SciPy, Numpy), for deep learning (e.g., PyTorch) and for computer vision (OpenCV) in their projects. Students may consult publications and online resources for ideas for their projects and implementation examples as long as they clearly cite references in their code and reports. When using such resources, it is essential that the students explain the contributions of their work in their project deliverables.

If students are combining the course project with the project from another class, the project proposal, milestones and final report should clearly explain what portion of the project should be considered for CPSC-459/559. Separate reports should be prepared for each course.