When the Defense Department created the new position of Chief Software Officer early last year, it was DoD’s first attempt to get a single official to ride herd over a vast enterprise that ranges from mainframes still running COBOL to DevSecOps pipelines to classified weapons systems, and everything in between.
For Jason Weiss, DoD’s first-ever CSO, there are at least a couple of big takeaways from having accepted that challenge: One is that there are a lot of pockets of the department where world-class software engineering is happening. Another is that what’s holding DoD back isn’t a lack of skill or dedication within its workforce, but rather, a lot of bureaucratic structures and habits that just aren’t compatible with modern software development.
Weiss, who will step down from his DoD job on Apr. 15 in order to return to the private sector, said he’s concluded there are two main factors that lead to successful software projects inside the department. Both have to do with leadership.
In cases where the military services have managed to implement modern software design practices, they’ve involved senior-ranking leaders who both “speak software,” and have the organizational savvy to maintain political support for what they’re up to.
“They understand the nuances of software and things like containerization and orchestration of containers, and they can bridge the gap between the engineer who’s actually doing the work as an individual contributor and the various oversight communities,” Weiss said in an interview for Federal News Network’s On DoD. “The second part is how suave that particular leader might be understanding that they need to create a groundswell of support, and fundamentally recognize when it’s time to compromise on something and add a little bit of overhead that might slow the process down in the name of moving things forward.”
But there are far too many programs that never even approach the point of compromising over small changes that lead to small delays.
Instead, they’re locked into acquisition mindsets that were originally designed for large hardware procurements: a list of requirements that must be met, and different colors of money for each phase of a system’s development.
Both of those concepts are terrible for software, which, unlike physical products, can be changed and updated in days or weeks.
“We have trouble reducing things into bite-sized tasks,” Weiss said. “We want to look at a set of requirements and say that all of these requirements need to be met, and as an organization, we’re not capable of effectively prioritizing them and recognizing that just because something has been deprioritized doesn’t mean that it’s not still a valid requirement. It just means that the warfighter has said, ‘Hey, I need this first and foremost, and I need this other thing second.’”
Congress has given DoD some room to experiment with using a single color of money for software development efforts. But lawmakers have only approved eight programs for what’s called the Software and Digital Technology Pilot Program; they declined DoD’s request to add several more in the 2022 appropriations bill.
Across the rest of the department, budgeteers, program managers and program executive officers still need to find ways to wedge software development into a funding system that was meant for carriers and tanks, with separate accounts for R&D, procurement and sustainment phases.
“When we look at the historical scaffolding that was put in place around the way the DoD procures systems, it was by and large hardware-centric, because you only want to create a keel on a ship once,” Weiss said. “But with software, it’s more like, ‘Oh gosh, that algorithm isn’t exactly what I need, I need to pivot that.’ That can be done in a two-week sprint. And I think that is fundamental to eliminating the color of money issue around software. And that conclusion was further codified with the ‘software is never done’ study from the Defense Innovation Board. Software is never done, so it never actually goes into sustainment.”
One of Weiss’s main tasks during his tenure as DoD CSO was to help develop what was originally supposed to be an update to the department’s cloud strategy, but was eventually renamed with a new moniker: the DoD Software Modernization Strategy.
Officials have said the new name reflects a realization that it needs to use cloud as a means to an end, rather than migrating systems just for the sake of migrating systems.
The new strategy makes a big deal out of the software factories that have started to permeate DoD, now 30 and counting, and aims to eventually reduce the policy barriers that are preventing the agile methodologies they’re using from just being the norm across the department.
Weiss said the quality of work he’s seen from those factories is top-notch.
“The ones I communicate with on a regular basis put out some amazing code — it’s state of the art, and it’ll rival anybody out there,” he said. “But I think it’s also important to recognize that in all of those cases, the industrial base plays a key role. This isn’t just government coders writing government code.”
In most cases, with the software factories — at least so far — contributions have tended to come from a lot of small businesses, with traditional Defense contractors playing a only supporting or coordinating role. Weiss said DoD will need to be careful about tailoring its relationships with vendors in ways that acknowledge that’s likely to continue to be the case, while also keeping the large prime contractors involved.
“With Platform One, there is no obvious prime contractor supporting it — it’s a large number of smaller vendors. And when they are onboarded, they’re actually paired with programmers from other organizations so that there is redundancy around that ecosystem,” Weiss said. “What’s important is understanding that we’re still going to need the hardware and the investments the large primes have made in highly-specialized labs, which justify their rates that they submit to the government. We still need that capability. So it’s going to be important for DoD and the industrial base to come together to find that correct balance between them, because we need both. It has to be a both-and conversation, not an either-or conversation.”
Weiss said another conclusion from his tenure is that his successors are going to need to have more authority in DoD’s organizational structure if they hope to make modern software practices more widespread.
He said Congress should seriously consider making the position a Senate-confirmed job, considering the amount of work that needs to be done to reform DoD’s practices.
“When you look at the amount of coordination that has to occur for something like a directive-type memorandum — and the time spent on that for something that’s relatively trivial — having an ‘honorable’ title to be able to go straight to a different organization and agree at that level is going to be vital,” he said. “I had very little influence in organizations outside of the DoD CIO, which is where my billet sat. That’s important, because the CIO can only give software a fractional bit of attention. They’re responsible for spectrum, they’re responsible for desktop services, they’re responsible for budget certification. That’s just the nature of the job.”