I was only involved in government contracting for a bit in the twenty-ought tens, but the whole bit about silly requirements and by picking the lowest bidder you were selecting the contributor that "often is the one who understands the technology least" gives me flashbacks to my time in government contracting.
I wasn't around for the whole JTRS fiasco, but from what I heard this effect played out there as well. Ridiculous requirements such as requiring only software updates to support new waveforms and restrictions on SWaP as well as unrealistic network sizes resulted in radios so expensive they couldn't be widely deployed. I know some people like to point to SRW as a success coming out of that, but it never has been successfully deployed to within an order of magnitude of the number of radios in the requirements and resulted in a third, incompatible, radio system used by infantry. Meanwhile, a company that was excluded from the program built the radio that the military actually needed, which could communicate to both of their existing systems.
I was around for it's successor where they swapped software for FPGAs and made the same "software only" update requirement. The primary winner of the contract had no experience in digital logic and decided to implement the thing in MatLab (which provided no real mechanism for multiple clock domains or other features needed for high performance large scale digital design). Other contractors had to fit their design into this mess of a system. When it didn't fit in any FPGA, they opted to put it in an ASIC. And another of the contractors managed to convince the PM that their newest dev board was perfect for integrating this -- allowing them to pay for the internal R&D for that board on the project's expense (after all PCIE and SRIO are practically the same, right?).
I left at that point because failure was the only option and I could make more money writing code to meet silly requirements in the tech sector.
The lowest bidder thing has always upset me. Either they are delusional or they are lying. If they understand the domain, they are going to sell you a system they built for someone else. Or they think the system they build for you can be resold to someone with deeper pockets. In which case they will push and push for you not to make it too niche.
Sometimes prebuilt is exactly what you need, but only sometimes, and that’s not what you paid for.
There are different solicitation types. Not everything is Lowest Price, Technically Acceptable (LPTA). LPTA is not usually the solicitation types for anything R&D. It's more for "commodities", like tires, or bids against things with very detailed specs, like construction/infrastructure projects.
That said, it's a fair criticism that procurement specs are often poorly written, and the evaluation processes insufficiently rigorous.
Maybe it's just where I've worked in the past, but "lowest bidder" contracts actually seemed relatively uncommon. Most big projects were "best value" (which still has some downsides)
The PMs only have a 3-5 year term. They can't be taken from the companies that bid due to conflicts of interest, so they rarely are in charge of an area they have expertise in. How long would it take you to get enough technical domain knowledge in a new field to actually see through a professional PowerPointer that's polished their technological turd for decades?
There's supposed to be a set of contractors that the PM refers to for technical advice that also provide continuity over longer terms, but the pay is not really competitive (due to budget cuts) so there's more churn and less talent than there used to be.
I'm sorry, but this is basically incorrect on every point other than the limited tenure. Source: I work in this sector.
PMs often come directly from industry. Once they do, there is a period over which they are not allowed to make direct decisions that affect their previous company, but the previous company is free to bid on that PM's programs and speak with that PM. When these conflict of interest situations arise, the office appoints a delegate PM to make decisions associated with that company.
There are absolutely varying qualities of PM, but the majority of them would be considered a domain expert or subject matter expert in some technical area. Standard practice is for a new DARPA PM to inherit existing programs that are already executing as the old PM finishes their tenure. When a PM leaves, the office tries to hand the executing programs to a new PM who knows something about that field (or is an expert). That doesn't always happen, for various reasons, but they do try to do it that way.
There is a large contractor staff that does a lot of heavy lifting on both technical and programmatic fronts. The PM is the decision-maker, but the contractor staff are integral. These contractor jobs can pay quite well for senior staff or technical experts. Maybe not FAMANG software developer salaries, but particularly for topics in the physical domain, the salaries can be significantly higher than the equivalent role in industry.
Once upon a time I knew a guy who claimed to know people who took the second lowest bid, figuring the lowest build was full-on crank and the second was only slightly unhinged.
I didn't know that guy long enough to find out how that worked out for his friend long-term.
Article is very interesting and focuses on the historic faces of DARPA, not really getting too much into the modern state of the agency. Over the past 10+ years DARPA have embraced the OTA - other transaction authority - as a contract vehicle to allow DARPA PMs additional flexibility and speed in letting contracts. This has allowed, perhaps, a return to more of the ideas of the old days in which a performer and PM might seek that "meeting of the minds" mentioned in the article, and issue funding to start work.
A real limitation perhaps that DARPA has is the 3-5 year PM timeline. PMs start with a project and vision for their work, often a multiphase effort. We often joke that the challenges DARPA offers are not all that difficult, only the schedule is. A few years ago I worked on what would have been a wonderful project on a 2 year timeline which was compressed to half that, with the usual shifting sands of requirements consuming the majority of the allocated time.
I can't say that giving PMs a longer timeframe would be a good thing - that 3-5 year PM clock keeps everything moving apace. Perhaps the longer-term programs ought to be given by the office directors to PMs, but then you have the disadvantage (mentioned in the article) of PMs having to execute a project they don't believe in. That hasn't worked well; disengaged PMs, focused on their passion projects, don't give the needed attention to inherited assignments.
The future of the agency is concerning. Given increasing time and funding limitations, coupled with the Heilmeier focus, there's been a tremendous push towards immediate results. Which means simpler targets that don't provide the amazing leaps ahead that DARPA was once known for. Look to their current SBIR topics, which include one on the use of language models for suicide prevention, and another on processing data to generate facility clearances. These are not leaps ahead, but instead incremental, focused improvements. We once lost a DARPA proposal because our plan was too certain to work, they wanted more risk. Quite a change.
DARPA still hires visionary PMs, and they have the tools again to realize these visions. I hope to see a return to the DARPA referenced in the early article. It will be fun to pitch ideas to those guys. I'm tired of talking to the ones who are too focused on today to think about tomorrow.
The few DARPA tasks I've worked on or had steerage over usually involved getting results quickly and iterating. The initial kickoff call with the stakeholders were about shared vision, but then there was a quick feedback loop until a final report was derived from the call notes and slides.
In that respect, it's a very decent way to do R&D - you have access to a highly motivated stakeholder with a clear vision of how your work fits in the larger scheme, and are allowed to iterate your way to a product both are proud of.
I was only involved in government contracting for a bit in the twenty-ought tens, but the whole bit about silly requirements and by picking the lowest bidder you were selecting the contributor that "often is the one who understands the technology least" gives me flashbacks to my time in government contracting.
I wasn't around for the whole JTRS fiasco, but from what I heard this effect played out there as well. Ridiculous requirements such as requiring only software updates to support new waveforms and restrictions on SWaP as well as unrealistic network sizes resulted in radios so expensive they couldn't be widely deployed. I know some people like to point to SRW as a success coming out of that, but it never has been successfully deployed to within an order of magnitude of the number of radios in the requirements and resulted in a third, incompatible, radio system used by infantry. Meanwhile, a company that was excluded from the program built the radio that the military actually needed, which could communicate to both of their existing systems.
I was around for it's successor where they swapped software for FPGAs and made the same "software only" update requirement. The primary winner of the contract had no experience in digital logic and decided to implement the thing in MatLab (which provided no real mechanism for multiple clock domains or other features needed for high performance large scale digital design). Other contractors had to fit their design into this mess of a system. When it didn't fit in any FPGA, they opted to put it in an ASIC. And another of the contractors managed to convince the PM that their newest dev board was perfect for integrating this -- allowing them to pay for the internal R&D for that board on the project's expense (after all PCIE and SRIO are practically the same, right?).
I left at that point because failure was the only option and I could make more money writing code to meet silly requirements in the tech sector.
The lowest bidder thing has always upset me. Either they are delusional or they are lying. If they understand the domain, they are going to sell you a system they built for someone else. Or they think the system they build for you can be resold to someone with deeper pockets. In which case they will push and push for you not to make it too niche.
Sometimes prebuilt is exactly what you need, but only sometimes, and that’s not what you paid for.
There are different solicitation types. Not everything is Lowest Price, Technically Acceptable (LPTA). LPTA is not usually the solicitation types for anything R&D. It's more for "commodities", like tires, or bids against things with very detailed specs, like construction/infrastructure projects.
That said, it's a fair criticism that procurement specs are often poorly written, and the evaluation processes insufficiently rigorous.
Maybe it's just where I've worked in the past, but "lowest bidder" contracts actually seemed relatively uncommon. Most big projects were "best value" (which still has some downsides)
The term is Lowest Price Technically Acceptable, gov't ought to be pickier (/have more domain knowledge) with what is acceptable.
The PMs only have a 3-5 year term. They can't be taken from the companies that bid due to conflicts of interest, so they rarely are in charge of an area they have expertise in. How long would it take you to get enough technical domain knowledge in a new field to actually see through a professional PowerPointer that's polished their technological turd for decades?
There's supposed to be a set of contractors that the PM refers to for technical advice that also provide continuity over longer terms, but the pay is not really competitive (due to budget cuts) so there's more churn and less talent than there used to be.
I'm sorry, but this is basically incorrect on every point other than the limited tenure. Source: I work in this sector.
PMs often come directly from industry. Once they do, there is a period over which they are not allowed to make direct decisions that affect their previous company, but the previous company is free to bid on that PM's programs and speak with that PM. When these conflict of interest situations arise, the office appoints a delegate PM to make decisions associated with that company.
There are absolutely varying qualities of PM, but the majority of them would be considered a domain expert or subject matter expert in some technical area. Standard practice is for a new DARPA PM to inherit existing programs that are already executing as the old PM finishes their tenure. When a PM leaves, the office tries to hand the executing programs to a new PM who knows something about that field (or is an expert). That doesn't always happen, for various reasons, but they do try to do it that way.
There is a large contractor staff that does a lot of heavy lifting on both technical and programmatic fronts. The PM is the decision-maker, but the contractor staff are integral. These contractor jobs can pay quite well for senior staff or technical experts. Maybe not FAMANG software developer salaries, but particularly for topics in the physical domain, the salaries can be significantly higher than the equivalent role in industry.
Once upon a time I knew a guy who claimed to know people who took the second lowest bid, figuring the lowest build was full-on crank and the second was only slightly unhinged.
I didn't know that guy long enough to find out how that worked out for his friend long-term.
Article is very interesting and focuses on the historic faces of DARPA, not really getting too much into the modern state of the agency. Over the past 10+ years DARPA have embraced the OTA - other transaction authority - as a contract vehicle to allow DARPA PMs additional flexibility and speed in letting contracts. This has allowed, perhaps, a return to more of the ideas of the old days in which a performer and PM might seek that "meeting of the minds" mentioned in the article, and issue funding to start work.
A real limitation perhaps that DARPA has is the 3-5 year PM timeline. PMs start with a project and vision for their work, often a multiphase effort. We often joke that the challenges DARPA offers are not all that difficult, only the schedule is. A few years ago I worked on what would have been a wonderful project on a 2 year timeline which was compressed to half that, with the usual shifting sands of requirements consuming the majority of the allocated time.
I can't say that giving PMs a longer timeframe would be a good thing - that 3-5 year PM clock keeps everything moving apace. Perhaps the longer-term programs ought to be given by the office directors to PMs, but then you have the disadvantage (mentioned in the article) of PMs having to execute a project they don't believe in. That hasn't worked well; disengaged PMs, focused on their passion projects, don't give the needed attention to inherited assignments.
The future of the agency is concerning. Given increasing time and funding limitations, coupled with the Heilmeier focus, there's been a tremendous push towards immediate results. Which means simpler targets that don't provide the amazing leaps ahead that DARPA was once known for. Look to their current SBIR topics, which include one on the use of language models for suicide prevention, and another on processing data to generate facility clearances. These are not leaps ahead, but instead incremental, focused improvements. We once lost a DARPA proposal because our plan was too certain to work, they wanted more risk. Quite a change.
DARPA still hires visionary PMs, and they have the tools again to realize these visions. I hope to see a return to the DARPA referenced in the early article. It will be fun to pitch ideas to those guys. I'm tired of talking to the ones who are too focused on today to think about tomorrow.
The few DARPA tasks I've worked on or had steerage over usually involved getting results quickly and iterating. The initial kickoff call with the stakeholders were about shared vision, but then there was a quick feedback loop until a final report was derived from the call notes and slides.
In that respect, it's a very decent way to do R&D - you have access to a highly motivated stakeholder with a clear vision of how your work fits in the larger scheme, and are allowed to iterate your way to a product both are proud of.