CMSC435 CLASS PROJECT

Last edited on 2024-01-04

The class project is intended to offer a substantive opportunity to apply the principles and practices taught in the classroom portion of our course. There are a number of teaching objectives. Some involve sharpening intra- and inter-team communication skills, some involve grappling with design tradeoffs, some involve validation, and more. In the present document, we will refer to the assignment as the project. Depending on assignment specifics, we might express this as either a problem to be solved or as a product space to be filled (a more entrepreneurial spin.) Ideally we will have an outside customer and identified product champion for each project.

Whereas small practice tasks done early in the semester are based on 4-person teams, each project will generally be undertaken by teams of up to 8 students who should collaborate and solve the problem together. Commonly each team attacks a unique problem.

Tasking is the mapping of students to a team project. We may designate a student to serve as the lead for administrative purposes; this helps us avoid confusion in starting up the effort. We will turn to these students as the points of contact. However, all team members share an equal responsibility in the project success, and there is no suggestion that one must work for or at the direction of another.

Students will work out responsibilities from that starting point. If any plan creates an imbalance in how much work is demanded of one or another team member, then possibly the team needs a better plan. (Welcome to engineering! Figuring out how to create effective plans - not just accept a default plan on autopilot - is one of the things we do.) In the same way, effort not being equitably offered by team members is an issue best addressed earlier than later. We need everyone engaged and contributing productively. (Welcome to management! Waiting until the end and then sorrowfully observing how some team members didn't pull their own weight in a failed project will win our sympathy, but not a good grade.)

A great deal of our course content involves finding ways to operate effectively as a team. We offer assignments that scale to the size of the team, so tolerating under-performing partners is a great way to increase stress and under-perform on the class project. In any case, no project will be re-scoped to adapt to team engagement, so the incentive is to engage everyone rather than carry someone as overhead. Earlier is always better than later for improving engagement, and that includes asking the instructor for guidance.

It is up to the teams to not only divvy up subtasks but also handle timelines. Typically we select projects based on knowing at least one natural componentization of tasking or tools. Teams would be wise to weigh many design alternatives before committing to any one. Still, it is the team's call. Our goal is to solve a problem, not just write a program. This class is about figuring out a solution approach, timeline, responsibilities, tools and a host of other things needed for success. That in turn requires personal initiative. Anyone who intends to wait for someone else to say Here, just do this... should not take this class.

Assignment

The suite of problems to be solved will be posted early in the semester. Teams will be formed and tasked as soon as we complete the scrimmage exercises which are conducted in the term's first two or three weeks. Each team should anticipate creating a charter and other planning documents to get moving quickly toward the major objectives.

1. Propose a solution and build it

The objective is to win approval to proceed on a specific solution path, which is to say, win a "green light to build." This is done by creating a proposal that persuades the instructor the team has a credible plan for how to proceed. A great deal of discussion appears in text further down on this page, but to call out important fine print: this proposal requires a pilot (technology demonstrator), an acceptance test plan that could only be passed by a system which meets our needs and an objective cost estimate of what it takes to get there.

Teams proceed according to their own timetables for completing a proposal, subject to the important reality that later parts of the assignment have hard deadlines for delivery. For example, the team must pass a critical design review on or before April 12, at which we will need to have seen sustained development effort. (An apocalyptic hackathon the night before CDR won't cut it.) As a practical matter, teams lacking a green light after March 15 will have a tough time arguing an effort is sufficiently sustained, and so will begin to lose credit accordingly. Afterr that I will define the goal for the team (at full loss of spec credit.)

Remember - a green light to build is predicated on seeing something function. We set a target of early March for the preliminary design review (PDR) and risk-reduction demos. We will talk about what this means in class.

2. Pass a critical design review and test with customers

After the proposal is approved, teams just follow the plan. The next milepost is a critical design review (CDR) that teams must schedule to occur no later than April 12, as noted above. Teams should show a complete system that is ready to proceed to acceptance testing with their customers. Based on what is shown, we will either approve going forward with acceptance tests or call out specific remediation (and its timeline) necessary in order to advance. Teams which have a rocky CDR may need to hold another review before being approved for acceptance tests.

Once past the CDR, we begin our "good to great" sprint. Teams will conduct acceptance tests in collaboration with customers and product champion, measure results and will improve the product based on observations made so at final delivery we know we solved the problem, not just hope we we solved the problem. To be clear, this is not when we build out functionality that should have been completed a month earlier.

3. Deliver

The final system package with all relevant documentation (as called out in plans and below) is due at SOB on the last day of class (May 9). There will be a fair amount of follow up assessment instruments (surveys and reports) harvested that day, so we ask that students plan the day accordingly. (We're engineers. We measure a lot, including about teams.)

That was the top level description of what this project is about. Now for a lot of tips, cautions and gory detail.

Regarding the proposal ...

This is actually the most challenging part of the class project - calling your shots. Historically, once they start working on it most teams take about three weeks to complete a credible submission. It is not a good moment to guess or make shallow decisions on what you intend to do. Perform your own research, interact with the customer to explore details of the space, discover the myriad of likely requirements, assess the risks of each of any possible design paths and more. The cost of do-overs may be higher than you can pay in the semester; use the course techniques to figure out best paths before you commit.

In many prior classes, someone else made the fundamental decisions about a program before students ever got involved. This was done when writing the assignment after which all students had to do is just write a program. In our case we need to figure out what is the right program to commission in the first place - one that not only solves the problem, but solves it in the face of hard and sometimes conflicting constraints. You only have so much time, you only have so much personnel, you only have so much experience, the target environment may only offer so much bandwidth or computational power, and so on. Which of possibly many solution paths will hit your mark?

Teams that merely echo back a problem statement by saying "yeah, we'll give you that" are doomed. Part of the instructor's mission is to ensure that each problem bakes thorny design tradeoffs into the interior, so pushing these off for discovery late in the semester is a great way to ensure you'll never have time to work out a real solution (much less pass the course.)

Let's be utterly clear: This is about finding ways to solve a problem, not just describing some program you intend to build. Software will certainly play a role in solving the problem, but the goal - and point of the class - is to learn how to commission the right software, build what you intended, recognize when you are getting closer to versus further from a solution, and then objectively demonstrate that the problem is actually solved. Writing the program? That's the easy part. Solve the right problem.

Important content you should plan from the start on providing: As already stated, your goal is to create a document which will allow the manager and client to agree to have you begin the build itself. There are a few things which will be necessary (even if not by themselves sufficient) for that approval.

• Delivery manifest. This is basically a checklist to summarize clearly what we're getting and how we will know we got it. It is used at delivery to, well, check that we got what you promised.
• Intellectual property statement. Make clear expectations for ownership or restrictions on this product (which thus includes understanding what limits are placed on components you rely upon in your build.) Note carefully how this information will affect your process. For example, data which are proprietary to the customer would not be something you could or would leave for team members in an open dropbox.
• Acceptance test plan. We will sometimes informally refer to this as the validation plan. The hard deadline for construction is as addressed later in this assignment statement, but you should pay particular attention to the fact that it is earlier than end of the semester. We insist on projects being built in time for them to be substantively used. (Yes, this is one of the ways we give you a chance to see the consequences of technical decisions you made earlier in the semester.) Thus, you should ensure that part of what you propose in the green light is a plan for how you will have the system validated independently after completion, and in time for a report of the exercise the delivered at end of the semester (deadline as noted.) We will surely talk a lot in class about how this is done!
• Cost estimate. Make sure you have a clear and supported prediction of what it will take to get you from the point of the green light approval to the walk through (i.e. when the build is done.) To be clear: this is not an invitation to guess. We don't want your opinion; we want a value that anyone would derive by using the same prediction method and model of your solution. The chief result we look for is prediction in hours. Show us you can research various prediction methods which might be named in our supporting class materials, and then apply them here. This component of your submission should be sufficient to support its use later in the second half of a cost exercise, where you report how much effort the project did take, and then explain why they are the same or different. Presumably you would not want to submit a green light proposal with a cost estimate which predicts there isn't enough time left available to actually do the job, or give a plan which predicts peak burn rates might occur at times when you are heavily conflicted with other course or life obligations. Plan! (Note: if some of the cost of moving forward involves funds, not just effort, then this is also an important piece of the negotiation process. Hardware, supplies or service purchases are possible, but not assumed, so be sure this is also part of what we all work out.)
• Timeline. We will want you to call out at least the major tasks, story lines or activities necessary to complete your project, and show how composing them should lead to success. Then, as you proceed with the build, we will want you to maintain (e.g. on one of your groups' web sites) a graphic summary of the status of completion, so we can track progress. Are you getting closer or further away from success? The 'manager' will want to know at a glance. (We will talk more in class about a variety of tools for this, such as JIRA or Trello. We hope you will come to see such tools as aids to success, not busywork or barriers. Learn to use these to improve your productivity.) The cost estimate should be consistent with the timeline; potentially you will reflect on how to create a Gantt chart.
• Press release. Yes, write the press release for your product first. We'll go over this in class. It will need to be a part of our green lights.
• Project advertisement plan. At final delivery we will want a description of the project suitable for showing what 435 students do in the class. Thus, it is part of your delivery obligation. Make sure planning for this is reflected in the green light.

Volume is not one of those "necessary" things in your proposal - precision is.

Lots of fine print:

• Some projects may have an outside client, and while we will certainly seek their input on proposals, the instructor will have ultimate signoff on what gets approved. Similarly, the instructor is solely responsible for determining grades, though it is fair to say that happy customers make it easier to determine good grades.
• When the assignments are made, a new directory in the repository will be made available to you, visible to all members of relevant groups. This is where you will make thoughtful decisions on how to craft product materials which are visible to others, and which will be one of the submission mechanisms at the end. This particular tool is useful as it also lets the instructor track code churn and related activities over time, and also sign off on teaching goals involving your ability to manage shared resources. Yes ... we get it that you like git. These are different kinds of tools. Let's move on.
• One of our goals for end of the semester may in some cases be to publish our product as open source; however, until that point you should not host intellectual property of our projects at other non-campus services without instructor permission. Disclosure of our IP to others outside the terms of our product definition would constitute a major violation of terms of this assignment.
• VMs are issued to teams and are available to you for purposes of development or testing as needed. Upon request we will re-issue 'fresh' VMs to you if something you need to test or demonstrate involves installation practices. You should actively manage all VMs to reflect best practices. If at some point earlier than the end of semester you are done with the VM, then let us know and we will secure it for safety.
• You should interact with the instructor throughout this process. Professor Purtilo is embedded with each team in order to ensure you are grappling with fine points at each step of the way. We'll give you all the feedback you like as you go along. That's part of the promise of concierge service. Because this will be an iterative process, you should plan to organize your document so the reader can readily tell what are the changes from a previous iteration. (Think about applying basics we talked about in class, right?) Sometimes feedback will be informal, sometimes it will be terse because we want to keep the pace brisk, and usually it won't repeat comments already made in response to prior questions, but you should feel free to get your money's worth on mentoring!
• You can have infinite free ungraded/informal iterations of your spec; basically we don't want any team moving forward with a build plan that we anticipate will take them over a cliff. At some point you will need to submit a document for grade. Email a notice to that effect, and don't presume that turnaround is instantaneous. The document which will be evaluated will be the file "proposal.docx" in the root of your project's repository folder. (You will similarly submit "coversheet-greenlight.pdf" or "coversheet-greenlight.txt" in the usual conventions. This document tells me who gets the grades, in addition to making the appropriate declarations about intellectual integrity and property.)
• Just because it is submitted for grade, do not presume that the document gets a green light; you may need to iterate further in order to reach accord. After approval is given to proceed, then the document can continue to evolve but changes will need reapprovals. You will be evaluated, in part, by consistency between your final delivery and the document above, so if the mission evolves without updating the plan, then you risk being gigged on the grade. One of the challenges is: taking enough time at the start to come up with an effective plan versus taking so long that you limit your development opportunity at the end. Hitting the hard deadlines for walk through and acceptance tests without an approved proposal in place is an automatic way to fail those later assignments.
• Important factoid: in the history of 435, no team has ever gotten a green light to build based on only one proposal submission. "One and done" is a process that works nowhere in the charted universe but introductory college classes wherein it is thought fine to presume the existence of unicorns and perpetual rainbows in order to bridge gaps in planning. And it shouldn't work there either ...
• Pro tip: We have yet to encounter the first team which felt they started working on the product definitions too early in the semester.
• Another pro tip: Sometimes a team will say "we didn't start earlier because we didn't understand enough about the problem to get going." Which is kind of sad. This is a capstone class where you are supposed to be practicing ways to figure out a problem. Waiting will not reward you with appearance out of the blue of a list of things to do so you can go on to get a participation trophy. If you find yourself thinking such words then it is a sign you should double the effort and exercise initiative sooner.
• There are two great ways to ensure your green light process is painful, drawn out, complex and jam packed with frustration. One is to submit a proposal that does not include analysis of alternate design approaches with an explanation as to why one has been chosen in particular. (Go back and read the part about "one and done" again in case that seems odd.) A second way is to reach this point without any working code, lo-fi renderings or pilot implementation. We don't just want an opinion about what to do; we want an informed opinion that gives confidence we can achieve a successful solution.

As noted earlier, each team will need to have shown a lo-fi rendering, pilot code, initial tooling or similar. We will discuss in class how these are risk reduction activities, and we'll work out at project tasking time some specifics appropriate to each domain.

Once you get a green light to be, go build it.

Regarding the CDR ...

We will conduct a critical review of your product in order to confirm that it is a complete implementation and ready for an initial deployment so you can begin the product improvement phase, which is based on user testing as defined in the green light agreement. You will present your work and demonstrate the system, and we will review code in detail; we will discuss issues and reach agreement on any last details going into the final phase. We will be looking for evidence of sustained development; implementation efforts that are needlessly compressed (i.e. have little time between the green light and critical review) will not get full credit on the walk through rubric.

The period between your green light and the critical review is when you have greatest opportunity to demonstrate technical prowess. You can largely complete this at your own pace, subject to the hard deadline called out here. All walk throughs must be concluded by COB on Friday, April 12th. This is a hard deadline. If you have no green light worked out sufficiently in advance of this date to offer a sustained effort on a substantive project, then this becomes a zero on these exercises. What is "sustained"? We normally measure sustained in weeks. Six weeks would make for a busy but reasonable time for two sprints, so picking up with your implementation earlier is a sensible target. Four weeks build time is a bare minimum but we know some projects need to adjust based on needs of outside clients, so we are open to working out details in the green light document. Teams without an approved product definition will have deadlines defined for them as the semester moves on; this comes with a corresponding loss of credit.

Teams are responsible for scheduling the reviews with me. We'd like to get all team members to participate if we can, but I do not require this because the logistics can make us all nuts. There are too many students/projects for us to do justice to checkpoints in class. Send the Provost a note sometime if you have opinions about class sizes.

During the critical review we will go over the project, evaluate what you show, open up code or designs to have you explain it, and much more. We hope this will be worth your while. To be clear on our expectations and pedagogy: if this is the only real team meeting we have in the semester then ... what a shame! I hold meetings at least weekly with successful teams. Commonly we bring in friends from industry (who are also veterans of our prior 435 classes) to provide a critique as well, though to be clear, only the instructor is responsible for determining your grades.

Please don't wait to the last minute and then panic if available meeting sessions are already used up; to quote something you will often hear said in industry, your failure to plan does not make for our emergency. Figuring out how to schedule a meeting, and then be prepared for it, is one of the basic practices that are part of the project expectations.

Remember complete means complete. Going into the next phase thinking that users will help you figure out how to flesh out code paths you should have built before this point is a great way to fail this course. The purpose of a critical review is to confirm that your product is ready to begin initial deployment (some would call this an alpha release) in order to discover opportunities to improve the product - not complete the product.

In my view, the time after your critical review is when the real lessons start, so if you simply halt effort at point of having a "complete" program then you are certainly not going to win best value from 435. Only used code is interesting code, and you're never going to understand how to design if you don't study the effects of your design decisions as seen via your users' eyes. Plan on maximum effort during the product improvement phase and you will win best value from 435.

At the walk through you will provide a signed cover sheet - hardcopy with original signatures - which I will use as the basis to give grades. Like all obligations, changes to this sheet later (such as students saying "ohmigod, I need to add my name!") will be treated as defined in the syllabus, which is to say, it is a late submission and gets graded accordingly. Our best advice on this point: figure this out in advance of the walk through.

Regarding deployment and delivery ...

The last phase of the project is deployment, to include conduct of acceptance testing. We fully anticipate that the initial outside use of your system will offer opportunities to improve the system. Do so. At some point you will have iterated through enough exercises to feel confident that the system is ready for acceptance tests, which we will have worked out in the spec. Conduct those tests.

To hammer a prior point: If you think deployment is the point where you get a passing grade by tossing a hackathon-generated program over the wall without quality improvement efforts - serious quality improvement efforts - then please do not take this course.

Your product must be complete by the prior review, so during this final phase you will be responsible for improving and then confirming the quality. A major step within this phase is to conduct the acceptance tests which were worked out in our spec. Perhaps you will have called out great tests and then constructed a product to exactly meet them without need of adjustment. Cool! You will be the first in our business to ever do that. Run your tests and hand the report in. The rest of us will measure and find ways to win even greater value for our customers, and adjust the implementation accordingly.

Your final project deliverable will be a report about the project. This is the point of delivery, so you will convince us that you provided everything that was promised. This is the point where another obligation called out in the green light (the acceptance test report) will be delivered. How did the project do? You proposed the yardstick, and we will have agreed after some negotiation. Follow that plan and report on it. Tell us honestly how things turned out. We will be especially keen on seeing the details of what you needed to tweak, fix, extend or refactor based on alpha deployment following the critical review.

An important piece of this deliverable will be a document which closes the loop on our cost estimations. Tell us what the actual effort turned out to be to get to the walk through. (Keep tracking effort during the acceptance testing. The final exam might reasonably ask a question about effort you had to expend on repairs during the tests.) Support this very carefully, and explain differences between the predicted and actual values.

The final version of your advertisement for the project (what you propose should be the persistent description of your product and successes) should be on the web at one of your VMs as of the final submission as well. Have it ready to port to our main server; your VM may go away following the semester, but a static capture of your site materials should be in your repository suitable for installation on the SEAM server.

The hard deadline for receipt of all materials (as called out in the green light) in the repository will by 0700 on May 9th. (Note, potentially we will have an in-class demo day in prior weeks so teams can check out what everyone else has been doing.) Delivery includes everything you developed, all the build commands and documentation, and the electronic version of reports and cover sheets. Insofar as you might have regularly maintained configuration control over your system on resources elsewhere, this is the point where you need to make sure we have your materials here. And please - do this sensibly. Committing a git structure that you shouldn't have been using anyway is not the right way to do things. Place the cover sheet for this deliverable by usual our conventions in the project repo folder as "coversheet-final.pdf" (or "coversheet-final.txt").

Please block out time on the afternoon of May 9th It is not only deliverable day and our last day of class, it is a serious assessment day. We will have many survey instruments to work through. (We're engineers, right? We measure stuff.) Make sure you will have time set aside to conduct these measurements with us.