DIB Guide: Detecting Agile BS

Version 0.4 Last modified: 3 Oct 2018

Agile is a buzzword of software development, and so all DoD software development projects are, almost by default, now declared to be “agile.” The purpose of this document is to provide guidance to DoD program executives and acquisition professionals on how to detect software projects that are really using agile development versus those that are simply waterfall or spiral development in agile clothing (“agile-scrum-fall”).

Principles, Values, and Tools

Agile adherents profess certain key “values” that characterise the culture and approach of agile development. The DIB developed its own guiding maxims that roughly map to these values:

Key flags that a project is not really agile:

• Nobody on the software development team is talking with and observing actual users using the actual code. (Note: PEOs and commanding officers don’t count unless they use the code.)
• No continuous user feedback (bug reports, assessments). One-time requirements gathering doesn’t count.
• Focus on “meeting requirements” is placed above delivering something useful quickly.
• Stakeholders (dev, test, ops, security, contracting, end-users, etc.) are working in silos.
• End-users are absent during development phases like release planning and UAT.
• Manual processes persist where automation (testing, CI/CD) is viable.

Common Tools Used in Agile Teams

Note: Tools are illustrative; no endorsement implied.

• Version Control: Git, ClearCase, Subversion
• Repo Hosting: BitBucket, GitHub
• CI: Jenkins, CircleCI, TravisCI
• Configuration Management: Chef, Ansible, Puppet
• Containers: Docker
• Orchestration: Kubernetes, Docker Swarm
• Issue Tracking: Jira, Pivotal Tracker

DevSecOps Stack Diagram

Illustrates modern tooling from plan, develop, build, test, secure, deploy, monitor, and scale.

Questions to Ask Programming Teams

• How do you test your code? Wrong answers: “We have a testing organisation”, “OT&E is responsible” Advanced: Tools used for unit, regression, functional, security, deployment certification?
• How automated are your pipelines (dev, test, security, deploy)? Advanced: CI/CD tooling? Infrastructure as code?
• Who are your users and how do you interact with them? Advanced: Feedback loops, issue tracking tools, comms back to users?
• What is your release cycle time? Advanced: Platform support? Containerisation? Config management tooling?

Questions for Program Management

• How many programmers are in the org that owns budget and milestones? Wrong answers: “We don’t know”, “It depends…”
• What dev/ops metrics do you track? How are they used?
• What have you learned in your last three sprint cycles? What actions were taken? Wrong answers: “What’s a sprint cycle?”, “Waiting on approval…”
• Who receives working code each sprint? Can we talk to them? Wrong answer: “We don’t deploy directly to users”

Questions for Customers and Users

• How do you communicate with the developers? Have they observed your teams? Asked insightful questions?
• How do you request features or report bugs? Do you receive responses? Are you involved in prototype feedback?
• How long does it take for your requested features to be delivered?

Questions for Program Leadership

• Are teams delivering working software to real users every iteration?
• Is there a product charter with clear goals? Do all team members understand their role in that vision?
• Is user feedback converted into backlog items within one month?
• Can teams modify requirements based on user feedback?
• Can teams improve their own process?
• Is the entire ecosystem agile, or is it bottlenecked by waterfall deployment?

If the team is truly working in an agile way, the answer to all the above should be “yes.”

Graphical Summary

Decision flow showing how to identify Agile BS: asking if teams deliver software, gather feedback, adapt process and requirements, and operate as part of a full agile ecosystem.

References

• DIB Ten Commandments on Software
• DIB Metrics for Software Development
• DIB Do’s and Don’ts of Software