PROBLEM
Originally, our client (NASA) provided us a problem domain to work with. The problem was stated as:
"How do we help systems engineers know when they're "done" with this stage of designing the rocket and, more specifically, understand the complex information required to do so?"
We conducted a good amount of generative user research and extracted some key insights.
- Engineers across levels don't share priorities, leading to tensions.
- Dependencies across disciplines often cause delays.
- Data is difficult to locate and communicate.
This led us to conclude that the greater problem was getting to "done", rather than not knowing when a phase was "done."
Over the course of ideating and concept validation sessions, we scoped this problem down further and narrowed it down to.
— How can we ensure design assumptions are shared between teams leading up to milestone reviews?
Why is this a problem?
Different assumptions about how the design should be done are only surfaced at milestone reviews and this causes engineers to go back and redo or restart their designs. Additionally, designs created using different tools might not integrate.
SOLUTION
At this stage, we have finalized a direction and have started designing low fidelity versions of this direction. We're currently in the process of testing out these versions with users at NASA to see if the direction helps in aligning design assumptions. Towards the end of July, the final solution will be up! Scroll further to read about our process, findings and ideas till now!
PROCESS
Overview of our Work
Background Research
Starting off, our team knew almost nothing about either space or systems engineering. We began with a list of questions to answer through background research, including the mission of Space Launch System (SLS), functional analysis and graph databases.
Analogous Domains
After developing a general understanding of the problem space, we were able to identify domains that share a similar context and workflow with NASA system engineers. We focused on fields where multiple teams collaborate to work against a wide variety of requirements in complex systems over time. We conducted 8 interviews with people whose roles ranged from NASA systems engineer to manufacturing engineer to construction manager.
To better understand our findings, we created journey maps to represent the processes, with a focus on their relationship to dependencies, collaboration, schedules as well as their pain points.
On-Site Interviews
We visited the Marshall Space Flight Center in Huntsville, Alabama and conducted semi-structured interviews and contextual inquiries with the SLS engineers. These interviews revealed contrasting perspectives.
We talked to both systems engineers (who focus on the integration and the interrelationship of different parts/disciplines) and engineers who are responsible for designing specific parts. These interviews revealed contrasting perspectives.
Sequence Modelling & Affinity Mapping
We sythesized our research by building sequence models (which helped us understand engineers' work processes and identify our knowledge gap) and an affinity diagram (which allowed us to uncover themes by grouping the data).
This process of affinity mapping and sequence modelling helped us identity some key insights.
- The adoption of new processes, systems, or tools at NASA is difficult, complex, and often unsuccessful.
- Engineers at different levels hold different priorities with regards to SLS.
- A formal process or repository of knowledge sharing was not observed.
- Interdependent processes run separately, resulting in redundancies.
Design Opportunities & Ideation
Every member of our team independently reviewed all of the research work we had conducted thus far, coming up with unmet needs and design opportunities. Synthesizing these identifications, we came up with the following guiding questions.
• How might we help engineers understand and visualize their part in terms of the SLS whole?
• How might we improve knowledge sharing across different teams?
• How might we enable the tools to adapt to engineers' existing workflow?
For each of these design opportunities, we used different rapid ideation methods such including Reverse Assumptions, 20 Questions, and Crazy 4's, to generate over 100 different possible solutions.
We affinitized these ideas to identify overarching categories and then narrowed to the top 4 that were the most interesting and impactful to out problem space:
- Promote knowledge sharing
- Align priorities across levels
- Track progress in configuration management
- Create context-aware tools that adapt to engineers’ specific workflows
Storyboarding & Speed-dating
We created a variety of storyboards that were used to conduct speed-dating testing to validate our user needs.
Based on our speed-dating exercises, we narrowed down to the top 2 ideas participants identified as more salient and impactful issues:
1) Prioritization Alignment
How can we better align different engineers' priorities and missions? Currently, Level 3 engineers tend to refine their work to a level of perfection that can cause delays, frustrating Level 2 engineers who manage higher level processes. The Level 2 engineers just want the work to be "good enough".
2) CONFIGURATION MANAGEMENT
Closing the loop: how can we improve the CM process so it is easier to track and follow a change as it is proposed, sent, received, acted upon, and ultimately resolved. This is currently a very manual process, which leads to gaps and detials slipping through the cracks.
Ideation & Further Narrowing
To encourage further divergent thinking, we conducted a creative matrix ideation exercise. This exercise allowed us to explore possible solutions to our design opportunities across platforms. This exercise enabled us to further identify design opportunities within the overall problem of prioritization alignment.
1) PrOACTIVELY ALIGN ASSUMPTIONS
Different teams' incompatible design approaches often aren't discovered until milestone reviews. Incompatible platforms prevent intergration and force completed work to be redone.
2) Enhance data consolidation
Data consolidation takes immense time and effort, given that data is spread throughout teams. This makes it difficult to communicate design decisions and justifications to other discipline teams.
3) Calibrate priorities across levels
When interdependent teams’ priorities are not aligned, seemingly small schedule delays can create a chain impact that cause overall larger delays.
Prioritization and Final Focus
We created 3 measures of criteria to decide our final focus area for the summer and plotted them in the form of a Prioritization Matrix
- Is this focus a rich opportunity that can be best solved through user-centered design?
- Will this have a strong, positive impact on improving work processes?
- Is this an actionable opportunity?
With these criteria in mind, we unanimously decided to move forward with the following refined focus: How can we ensure design assumptions are shared between teams leading up to milestone reviews?
CURRENT & FUTURE WORK
With our final focus established, we're moving into the summer with designing low fidelity features for our final solution. These features will be tested to validate its usefulness. Post this, we plan for the features to be integrated into a single platform that fits in with engineers' current workflows to ensure that adoption is not an issue. Visit again by the end of July to see our final designs!
