CMSC 435 PROBLEM STATEMENTS
Last edited 2025-02-12. Projects below tasked in Spring 2025 semester.
Timelines for the semester will be as called out in a separate document. How a team establishes its intermediate goals in order to meet requirements and hit hard deadlines is up to the team. I suggest that you conduct a risk assessment right away, but how long you put off discovering thorny issues is also up to the team. You can even treat the project as a big hackathon at the end if you like; it's your career, after all, and it isn't like campus doesn't promote such things. But, I bet you'll also find that hackathons are better for campus than for you, and in any event this approach has yet to end well in a 435 project. Your call.
All students on a project are equal stakeholders in the effort. No one person must work at the direction of another; we cooperate in order to win. The incentive to hold others accountable is clear: all of these projects are scoped with the expectation we'll have full effort from everyone, and there is no partial credit for partial success. We tolerate others' lack of engagement at a cost paid in our own time and grades.
The incentive to fully participate should be clear. First, the final exam is constructed to reward those who have done the work all along. It chiefly address deep technical issues involving this project. Those who didn't do the work won't know how to answer questions and pass the class. Similarly if we have done the work, but not established a paper trail to this effect along the way, then we won't be able to support answers to our questions which require reflection. There is only one snapshot in time from which to work - the end of project. Without the paper trail to analyze what the perspectives were historically, then there is not much credibility in the answers. Said another way, nobody has yet been able to recreate a project record that is credible enough for exam answers, and of course there is no ability later to go back in time to insert material into the record. (Translation: Waiting until near the final exam in order to manufacture history for convenience of answering questions is deemed "not credible".)
Second, the cover sheet submitted (as our academic integrity and intellectual property statement) lists who gets credit; no name, no credit. The decision of who signs the sheet is ultimately a team consensus. Basically the rest of the team can vote someone off the island, though this is not the common occurrence, and we'd like to have exhausted our inventory of practices to promote positive engagement before it reaches that point.
My advice: do the work and document it to pass the class. It might just be that these practices actually work too. Bonus!
I offer these projects as an opportunity for us to practice substantive application of software engineering principles. We will learn by trying them out, making design decisions and then studying the nuanced consequences. We can't close that loop if we don't have a detailed record of the decisions we made, however, and that is the most important reason for our serious obligations to log activity and articulate our reasoning as we go. Working code alone won't tell us we reached our learning objectives. Please take this process seriously from the start and we will win best value from 435.
-- Jim Purtilo
1. ANTS
Long form: Automated Identification and Annotation of Movable Joints in 3D CAD Models using Vision-Language Models (VLMs)
Current era digital technologies have been a boon to scientists who are now able to create sophisticated models of their subjects from nature for detailed study. This is especially true for entomologists who can learn much from these studies, and of importance, also apply much. Whole new avenues of investigation into, say, locomotion and robotics can be opened by study of ants, for example. Engineers are eager to learn more about how nature managed to do it.
The problem however is that creating these detailed models is labor intensive, especially for high acuity systems where (as an example of interest to us) we must capture the joints in insect limbs accuractely to be of any use. The axis of movement, range of motion and more are critical.
The research opportunity is to develop a software pipeline that leverages Vision-Language Models (VLMs) to analyze 3D CAD models of these subjects (ants), identify movable joints, and generate an annotated 3D model with labeled articulations. We'd like if system will facilitate automated recognition of mechanical components' degrees of freedom, enhancing efficiency in robotics, simulation and digital twin applications. We'd like if that is true and we can reduce the cost of creating such models ... but we don't know. This project is intended to let us find out.
To keep this project doable, we've worked out the likely technical approach.
This should produce a software tool capable of analyzing and annotating mechanical joints in CAD models, where our initial domain will be of modeling ants, though the design should not limit us in application to other domains. The system would offer an interactive interface for verifying and refining detected articulations, with export of models compatible with robotic simulation and kinematic analysis. So yes, in the end, our AI system should allow scientists to start with images of ants and create animations that teach them how to walk correctly.
This project is to engineer an effective tool to enable scientists to conduct research. There is a fair amount of ambiguity involved and the mission will certainly evolve. Such is the nature of research. We will nevertheless recognize success when we are able to observe scholars prepare accurate and dynamic models by mostly automatic means, and at substantially less cost than by manual techniques. There is publication potential in this work and its many follow on tasks.
Our clients in this project are Prof. Evan Economo in UM's Department of Entomology (who can tell you anything you want about ants!) and Dr. Xiaomin Lin at JHU (and expert in imaging and ML tech.)
2. PLANTS
Commitment to environmental sustainability is enhanced when people have strong connections with other species. Strong connections with nature, in turn, enhance human health. The campus landscape supports a diversity of plant species that provide rich opportunities for developing those connections. But for most people those plants are little more than a green blur through which we rush as we go about our daily lives. Few students know they are in the midst of an Arboretum and Botanic Garden. Fewer know that the plants that surround us have fascinating stories to tell. Plants create the oxygen that allows our very existence. The food and fiber they provide have driven the development of society and culture as we know it. Plants also enrich our intellectual, aesthetic, and emotional lives, having played fundamental roles in arts and literature throughout human history. Plants are the source of pigments in paintings, dyes in textiles, subjects and symbols in literature, and form the very pages on which books are printed.
To increase connections to the UMD landscape and foster appreciation for the biodiversity of Maryland by sharing the stories plants have to tell, the Spring24 CMSC 435 Plants group developed the Roots and Routes web tool. This tool pulls location information from the Arboretum geographic information system database to show users the plants near them as they move around campus. In addition to the names of the plants, users have access to narratives from a range of interconnected perspectives including ecology, literature, art, chemistry, and history. Information is conveyed in multiple formats including straight narrative, quizzes and games. The ability to earn "seeds" for engaging with plants entices users to repeated engagement.
Based on user feedback, we learned how challenging it is for users with no plant knowledge to identify which plant is which, especially when there are many plants around them. We have a great base, but we needed to enhance the map tool. In Fall24 we thus set about implementing these enhancements by adding image recognition capabilities to Roots and Routes. Imagine pointing your mobile device's camera at a plant on campus getting the correct identification with a link to all the information Roots and Routes has for that plant.
We got close! The framework and human factors in the new app seem right, but AI under the hood wasn't giving us the accuracy we need for this to fly at scale. It was a good pilot but we need a do-over on the core machine learning component. That's our task this semester.
The problem to solve is providing a high-accuracy plant identification system that works at scale. To work at scale the system needs to work on all trees and shrubs planted in the UMD Arboretum and Botanic Garden and it needs to be amenable to automation so manual image processing does not create a bottleneck. We are not reinventing an entirely new app for this, rather, we are going to focus on the core AI in this effort, and simply ensure that the design will allow us to drop it into Roots and Routes. We will be demonstrating Roots and Routes for visitors on Maryland Day and ideally the AR capability is part of that demo. This needs to be a robust and polished product which reflects well on the campus.
There are many potential solutions to this problem and the team should plan on performing a substantial amount of experimentation with different techniques in order to zero in on the winning algorithm.
Remember, we are crafting an important user experience, not just making a location-aware/AR application that dispenses plant information. Overall we will recognize success when we see measurable improvement in the levels of engagement with the plant scene on campus, consistent with the abstract goals called out above. The successful AI component is just one piece of the puzzle to get right along the way.
Our client in this project is Prof. Maile Neel in the Department of Plant Science and Landscape Architecture as well as the Department of Entomology UMD.
3. XAI
The Neuromotor Control and Learning (NMCL) lab is the home of a research group at UMD which examines the human cognitive-motor mechanisms when individuals collaborate with AI-driven humanoid robots having adaptive planning capabilities. One experimental research platform they use is a VR system called VTEAM which enables the team to conduct experimental studies where humans complete collaboratively a task by taking turns with a simulated humanoid robot (Baxter, Rethink Robotics) controlled via an adaptive planner (BFS). VTEAM has an interface (previously developed in capstone projects) which allows us to flexibly parametrize characteristics of the task and of the robot. (See Jayashankar et al., "Assessment of a Novel Virtual Environment for Examining Cognitive-Motor Processes During Execution of Action Sequences in a Human-Robot Teaming Context", Lecture Notes in Computer Science, Springer, vol 14694, pp 147-166, 2024.)
However, one issue with VTEAM is that often humans cannot understand the decision-making process of the AI system driving the robotic teammate which appears as black box to humans affecting thus their behavior and trust on how the machine operates. A solution to this problem is to include an explainable AI component (XAI) which allows the AI of the robotic teammate to explain its behavior.
VTEAM thus needs to be expanded to incorporate an XAI capability. Moreover, the experiments to be designed will require the XAI component to be parametrized in multiple manners to manipulate when the explanation is offered (before, during, after the robot executed its action), the frequency at which the explanation is provided (every turn, every other turn, every X turn, on demand), how the explanation is delivered (text on screen, synthetic voice, graphic illustrations), and probably a number of other ways not yet anticipated.
The team will need to grapple with legacy code from the prior 435 team's effort, and add appropriate hooks into the run time system, though as much as possible should work to craft new capabilities into separable components to increase flexibility and reduce cost of maintaining these tools. However, a fundamental design question which this project team will need to address is also where the explanation is offered - entirely within the VR tools or as an additional output channel (e.g. separate screen or other output devices.) This could affect results. Piloting will need to be done with the research team.
Overall our task is to provide the NMCL lab with a software solution that fully implements the parametrization of an XAI system based on the designs to be fleshed out collaboratievly. To be clear, development of the XAI algorithms per-se is not the 435 team's prime focus here. We will pilot some methods in order to demonstrate the power of our system, but instead the real goal is to augment the current interface of VTEAM to enable flexible parametrization of the XAI capabilities. This will allow diversified experimental manipulations of the XAI to address a broad range of problems and questions in human-robot teaming. This is indeed a tech project in support of conducting research. Ambiguity abounds. We will however recognize 435 team success when we see the NMCL research team able to efficiently conduct studies which rely on XAI-augmented tools.
Our client in this project is Prof. Rodolphe Gentili and his NMCL lab, for which the points of contact on this project are Jayesh Jayashankar and Hunter Frisk.
4. PESTICIDE APP / 5. PESTICIDE DATA
Healthy crop production is critical to the economic and social wellbeing of our country and world. Unfortunately, due to high pressures of insects, weeds, and diseases, pesticides are often needed to protect and produce quality food. Pesticide usage is highly regulated and complex and requires special licensing and education to prescribe and apply pesticides in the United States.
There are also over 7,000 currently approved organic and conventional pesticides registered for use in the United States, each of which has a PDF label of application instructions and safety information (some upwards of 40 pages). Information on these labels is critical to the correct regulation, application, and safety for farm workers. However, the structure of these labels has not been regulated, and farmers, researchers, agricultural extension workers, and regulatory agencies manually scrape pieces of information from these labels, often making mistakes. This leads to confusion, incorrect choice of pesticides, and off label usages which can be detrimental for farm workers, consumers, and the environment.
Our opportunity is to create an uniformly-structured, easily-searchable database of the pesticide information and offer it via a webapp that can be placed in the hands of farmers and other stakeholders. The challenges however are substantial. While a database of accurate pesticide information for every crop would be highly impactful and valuable, compiling these data from diverse base material by traditional techniques would be expensive and prone to error. This suggests that we leverage AI methods for data ingest. And because this sort of tool has not been done before at scale, an additional challenge will be to model the work flow of stakeholders and ensure a suitable front end to these data will be inviting to all. Requiring excessive manipulation of filters in complex searches in order for the users to find just the right data can make the system just as unusable as the present system, which requires manual hunting through different data sources.
The system envisioned would enable filtering pesticides by crop or target pest, or by organic/environmentally friendliness. It would enable tracking of pesticide applications, and ensure postage of appropriate usage information for worker safety purposes. It should facilitate connection with the original data sources so material can be directly checked by stakeholders. The system should faciliate streamlined and timely management of the database as pesticides move through the registration and deregistration process of the US EPA, which is the agency responsible for evaluating safe and effective use of pesticides. The US EPA website hosts a weekly updated list of all currently registered pesticides in the US along with every pesticide label. The information most helpful for farmers includes: product name, active ingredient, company name, EPA reg no., first aid information, protective equipment for application and handling, restricted-entry interval, pre-harvest interval, approved crops, target pests, and pesticide rates. We anticipate that over time stakeholders will appreciate being able to search on additional parameters, so the prompting techniques (and data schema) should be readily extensible.
There are two fundamental tasks in this project - app creation and database construction - and while they are both essential to system success, they are also suitable for separate tasking so long as we ensure they work well with one another. We thus anticipate tasking two teams to this project, called Pesticide App and Pesticide Data. It will be the mentor's role to facilitate coordination between these activities.
Our client in this project is Dr. Scott Cosseboom, a scientist at Cornell.
6. LEXICOGRAPHY
Echtralex Contemporary Lexicography is a private, for-profit corporation with a product line that addresses the People's Republic of China. Echtralex's work promotes accountability and transparency, protects democratic values and ideas, supports the efforts of independent media, enhances the freedom of information, improves democratic governance and political processes, and facilitates more productive communication between countries.
Our flagship product is a book called A Chinese-English Dictionary of Current Events in PRC State Media. The dictionary is compiled from a proctored selection of publicly available information distributed by a number of PRC government ministries. Updates are released every two weeks in the form of a large PDF. The full text is downloadable through our website (where it is searchable as well) on a delayed release schedule to encourage users to pay for up-to-the minute access. The PAI that serves as our source material is likewise readable (and searchable) through our website. Documents are set in parallel (Chinese-English) and are available the instant they are uploaded.
These products serve the needs of translators and cryptologic linguists, policy analysts and researchers, and foreign service officers and diplomats. They are currently used by analysts at the US Department of State, the Congressional Research Service, the Library of Congress, the National Security Agency, the Central Intelligence Agency, and the North Atlantic Treaty Organization. They are likewise used by members of the media, China watchers in dozens of countries, human rights activists around the world, educators and academics at institutions such as CSIS and SAIS, and language learners now studying at institutions across the United States.
Echtralex is in need of a software solution that will improve our workflows when it comes to preparing these bi-weekly PDFs. The processes now used are onerous, inefficient, and involve many different steps across several different software packages including FilemakerPro, Nisus Writer, Microsoft Word, and BBEdit. What's more, there are always unsightly mark-up issues that turn in the final stages as the processes run their course. And ultimately there are a number of bells and whistles that we would like to add to enhance the appearance of our product. The goal is to reduce the production and compilation costs of this product, eliminate defects that might crop up from manual editing, improve product consistency and perhaps enable additional use-cases for these data along the way.
Our client in project is Echtralex, for which the point of contact is Michael Horlick.