Purely fictitious for the fun of exploration, this submission explores what it looks like when vendors of a product and/or service effectively hold a customer in a form of stasis - never really delivering, never really finishing, and never really providing enough understanding for the customer to be happy, sad or equipped to decide to stay or leave - just unfinished, bleeding money. Because of the size, this post is broken into multiple parts.
Continued from Part IV:
From: CTO, Senior Technologist Auditor, and Business Sponsor
This brief covers the recently canceled BioReCo project and identifies two primary elements:
a) What are the top 10 things we did not do correctly (observations)?
b) How will we make sure we will not do them like this again (recommendations)?
It should be noted that we arrived at seven, rather than ten, primary items that we observed and recommended attention going forward. It is our belief that by addressing these seven elements, we will equip ourselves for greater success going forward.
Please address us directly with questions. Thank you.
Observation 1: The Project/Business Sponsor did not know how to manage the project
The Project Sponsor, while being responsible for defining the business problem and the manager of project funding, did not understand how to appropriately manage the finer technical points of the project and therefore could not.
In particular, the Project Sponsor's strengths are to identify the business problem to be solved and then verify that deliverables do in fact not only meet expectations, but actually solve the business problem. Second to this, the strength and role of the Project Sponsor is to, just like normal daily business, manage planned to actuals comparisons, validate forecasts, and agree to or decline funding for additional project work.
In this particular project, most of the work and staff was technological in nature and required a technologist, working on behalf of the business, to verify that solutions were necessary, appropriately, timely and complete. For this project, we believed the solutioning vendor was would fulfill this role - acting on behalf of the business and helping the business solve problems. However, what we discovered was that the vendor did not fully understand the business problem, but did in fact understand the technology and the solution was never proven to be useful to the business.
Recommendation 1: Pair Business Sponsors with Technology Sponsors at project inception
For any and all projects in the future where there exists a business problem to be solved allegedly requiring a technology solution, assign an corporate employed executive level technologist to the project for accountability to right technology and financial solutions in accordance to stated business problems.
In other words, when assigning a business sponsor, also assign a technology sponsor to be paired into the leadership structure. Furthermore, associate financial incentive for the technology sponsor to meet the business problems as defined by the business sponsor in as quickly and economically responsible manner as is reasonably possible without compromising stated business and customer quality expectations.
Always pair business and technology together at the leadership/sponsorship/problem definition level and lead the project from the 'paired' top down.
Observation 2: Project deliverables were not predictable, tangible, or comprehensible to all involved - in particular the Business Sponsor funding the project
Nearly all deliverables were stated to be important to the business, the business problem and the subsequent project. However, it seemed that only other technologists could actually appreciate, understand and actually use the deliverables of the project.
Furthermore, most of the project milestones were technical in nature and therefore under-appreciated by non-technical staff. In other words, when a technology team would be happy and communicate success about a particular milestone suggesting the project was making great progress, non-technical team members and especially the business sponsor, could not translate the value and therefore could not reasonably agree that the progress was useful.
Recommendation 2: Decompose the project and deliverables down into parts and pieces that are not only tangible and make sense from the Business Sponsor's point of view, but deliver them in a predictable and repeatable rhythm across time
First off, we recommend that all project milestones and deliverables are actually defined and stated as business objectives, not technical objectives. Given the reason the project exists is to solve a business problem (i.e. launch a new company), it make sense that there must and will exist technical tasks and milestones along the way. However, in order for the business to define a project expenditure as successful, the objectives must be stated in terms understandable and measurable from the perspective of the business. You want $10? Deliver 10 business objectives. You want $10 more? Sign up to deliver 10 more business objectives.
Second to this, what gets delivered and how often must assuredly make sense from the perspective of the technologist - but must more importantly make sense from the perspective of a business person attempting to solve a business problem. In other words, do not deliver only a portion of one objective, deliver the whole objective in one sitting. Is the objective statement to broad or big to be delivered in one motion? Then break it down into parts and pieces until such time that each objective statement _can_ be delivered in one motion.
And what is 'one motion'? A predictable, repeatable delivery pattern that the technologists must strive to practice. In other words, on the business side of the house, deadlines are expected to be met regardless the situation and it should be no different on the technology side of the house. For example, were we to expect usable, tangible, measurable and useful software deliverables every thirty days the answer from the technologist is not "it cannot be done", but rather offering a counter-proposal suggesting that they will break the objectives down into parts and pieces until they identify what they _can_ deliver inside thirty days.
Observation 3: Not all deliverables created for the project were actually useful to anyone
We already discussed this to some extent, but the summary of the conversation lay around deliverables, their usefulness, and determining whether money should be spent to develop them.
When spending someone else's money, it seems nearly all deliverables are important to somebody. However, many of the deliverables announced as 'successes' were in fact only useful to one or two other people, or only useful to other technologists. So in effect, the business was funding deliverables that it could not use, but some were touting as very useful.
Recommendation 3: Minimize and/or eliminate the creation of deliverables that do not first have a customer and purpose to exist
Our primary recommendation is couched in the need for all measurable project objectives to in fact be stated business objectives. If a deliverable does not or cannot be directly traced and evidenced as helping to solve a stated business objective, it should not be created.
Second to this, associate business objectives to deliverables, and deliverables being in a useful, measurable, tangible form, to further funding. In other words, rather than signing a fixed timeline, fixed bid/cost contract across time and then blindly paying invoices as they arrive - only pay invoices after business objectives for that period have been demonstrated to be met and the deliverables are in fact useful for meeting said objective.
There should be a direct trace between deliverables and business objectives in advance, not arrears.
Observation 4: Project Statusing was inconsistent and non-meaningful
Project status reports often included subjective statusing such as "people are working really hard" or "we found some problems, but they are being addressed". Furthermore, some of the status reports measured progress by counting the number of file checkins to the code repository system, measuring defect founds (but not fixed), and the number of meetings that had occurred during the week.
An additional challenge lay in the fact that there was really no predictable pattern of when actual solutions would be delivered, but status reports were delivered in a predictable and repeatable pattern suggesting to us that there was perhaps more time spent talking about doing work, than actually doing work.
Recommendation 4: Measure progress by the number of tangible, tested, business functions from the perspective of the Business Sponsor, not the technologist or Project Manager
Put solution delivery and statusing on the same delivery schedule so that the solutions and reports of solutions share common timing.
Furthermore, make the measure of 'progress' the actual delivery of tangible, tested business objectives as stated at the beginning of the project, rather than determined in real-time and communicated ad hoc through status reports.
In other words, other than the small joys of personal and professional fulfillment along the journey of delivering, it does not matter what the Project Manager, Technologists or any other members of the solution team think or characterize as 'success' and 'completion' - it only matters whether a business objective has been met, can be demo'd or shown, is actually tangible and verifiable by the business sponsor, and actually knocks one objective off the list as 'done'.
It should be noted that at project charter (outset or beginning), the definition of 'done' for any objective should uniformly be defined as: known necessary coding is complete; known necessary testing is complete (unit, integration and functional). Acceptance testing will actually be performed by business people, appointed by the business sponsor, to verify that what is delivered and communicated as done, actually meets the need.
Observation 5: Staffing and costs grew reactively
This particular observation contains more than one point, but we believe the pivotal conversation point revolves around deciding what kind of staff is necessary, for how long, and how.
We believe our philosophy of hiring needs to change for the future. Historically, we have sought the best bid from vendors by looking at hourly rates first, timelines second. We felt that, being reasonably intelligent people, we would be capable of identifying a vendor that would provide good quality output balanced with the needed time and cost. In truth, we did not have good measures in place telling us the type of roles and responsibilities we truly needed to fill, let alone the definition of successful progress and associated 'quality' output. In fact, we found our definition of 'good' was not only different than that of the technologist vendor, but our definition of 'good' changed as we had more time to think about it.
As a result, given our definition of 'good' changed through time, as well as, our lack of clarity regarding all required roles and responsibilities to do the work, when our timeline began to slip we truly had no bearing as to whether staffing levels were correct, and whether staffers themselves were the correct staff. Thereafter, anytime someone made a request for more staffing we really had no feel for whether it was necessary at all and whether the additional staff would or would not positively impact the solution timeline and quality. As the vendor itself began to believe the answers was 'more staff', we had no real choice or context other than to approve increased staffing levels every single time they were requested.
Recommendation 5: Hire only a few really good technologists/solution providers and stick with them from end to end rather than throwing tens and hundreds of lower cost, lower ability resources at the project reactively
First, define a single problem statement and the associated business objectives that must be met in order to define success for this project effort.
Second, for each objective define a definition of 'done' that makes each objective tangible, measurable and complete from the perspective of a business stakeholder. In this example, said 'definitions of done' are more like acceptance criteria or otherwise criteria by which a business stakeholder decides whether a deliverable actually solves the need/meets the need.
Third, ascertain the roles and responsibilities of people we need on a team in order to meet these objectives (with definition of evaluation and acceptance) and solve the problem statement.
After having a definition of the problem statement, objectives, associated definitions of done, and asserted roles and responsibilities to perform the work - the next problem we need to solve is somewhat subjective, but worth discussing ... do we hire N medium skilled workers at a medium rate in order to get the work done, or do we hire N-1 high skilled workers at a higher rate to get the work done? There is no clearly objective algorithm, but it is our assertion based upon this experience and others that more people do not equate to higher output. Furthermore, we believe that having fewer high skilled people may actually improve output quality, communication and teaming.
It comes down to a choice on the part of leadership. Going forward, we believe we should experiment with having smaller teams/fewer people who are more highly paid and have higher senior level skill-sets than trying to hire N+ people at a lower rate and increasing the probability of communication problems, complexity, latency and overall management challenge. The responsibility lay with us, the customer, and never the vendor. And were we to take on a 'partner' versus 'vendor' in the future, our responsibility does not change - it is our dollar and our responsibility to define how we'd like to spend it. Absent such clarity, we will most likely repeat history.
Observation 6: Teams created according to skill-set/role contributed to non-contiguous system-level decisioning and solutioning thereby increasing costs, time utilization, and a higher probability of defect insertion and complex causal analysis
Summary? We inadvertently contributed to silos. In the beginning of the project it appeared to make sense having a research and development group headed by a senior architect who did all of the hard thinking and then passed it down the line to everyone else. Furthermore, it appeared to make sense to have the requirements people working together, the human factors/usability people together, and so on. What really happened, coupled with the fact that neither the project sponsor, nor project manager could manage to the technical details, was that teams focused on their problems uniquely and did not collectively work towards single goals as defined by the business. Explained: Absent a clearly defined set of business objectives met by pre-defined technical solutions, each group of roles/people on the project interepreted the business objectives differently and pursued what they considered to be the goals for their particular skill-set/group.
Now, we already identified that we should have had appropriate pairing with senior technical resources who clearly understood the business objectives (as a method of mitigating over-engineered solutions); and we already noted that had we single business defined mission statement decomposed into business objectives which would be met by technical solutions we would have facilitated a common goal for all project participants grounded in business needs, definitions and measurements of 'done' or 'success'. Absent these two elements, and then by grouping people together by role/skill-set we inadvertently contributed to many groups of people individually believing they were working together when in fact no one really discovered they weren't working together until downstream in the project when parts and pieces did not fit together in a usable manner.
Restated: Absent business derived mission and objectives coupled with people grouped according to their department instead of individual business problems, we created very effective silos and prevented system level, cohesive, tangible progress from the perspective of the business.
Recommendation 6: Make self-contained teams responsible for all aspects of delivering a tangible business function from design through testing, development, delivery and Business Sponsor acceptance
In the future, regardless the history, lobbying, politic or predispositions, we need to practice creating self-contained teams of people who will deliver, from end to end, one complete business objective at a time.
In other words, the business provides US$1.00 and expects to receive one business objective that is completed, working, tested and validated as useful by the business representative setting the expectation from the beginning. The measure of success from the perspective of the business is when each and all business objectives are delivered - not when technical things happen along the way that are meaningless to the end-user.
It is our explicit recommendation that going forward projects may contain many teams, but each team contains all role types necessary to deliver one business objective. For example, one team may contain a designer, developer, tester, business representative, data person and perhaps usability engineer when necessary. This team would deliver to us a tested, complete, usable business objective with all design, development, testing, usability and business validation completed in one motion. In this manner we are assured the possibility of putting in US$1 and getting back out what we ordered for US$1.
Anecdotal note: There may be special times when it makes sense to co-locate same skill-sets for thinking/designing, R&D, etc. However, regardless preferences or opinions, we no longer believe, nor support, putting together like skill-sets in co-located groups. Going forward all teams will be hybrid, self-contained, business objective driven teams.
Observation 7: Project Management did not actually facilitate success by definition of the business, but by definition of the project
We believe the project was staffed incorrectly in context of project leadership. Our historic world view has been to assigned a Project Manager from the PMO every time we have some type of activity called a 'project'. The nuance we have discovered is that in some cases, when the project manager did not really understand what was occurring on the project, she put in more controls on the teams making them focus more on process than on actually delivering work. At other times when the project manager did not fully grasp what was going on or did not fully understand the weight/impact of a system level situation, not wanting to appear ignorant or unable to handle things, this person seemed to blindly support whatever the technologists were saying.
In both cases, this challenge seems to suggest that when delivering a software and/or hardware system in particular, the problem to solve may not be so much putting a "PM" in charge of the project, as much as, having someone in a leadership capacity that has technical or quasi-technical experience in these sectors of technology, as well as, an ability to be a respected leader in the language of software/hardware engineers in alignment with business goals. A difficult task to be sure; but we now challenge our historically practiced logic that if someone is certified as a project manager they will be able to successfully navigate the waters of technical and business objectives successfully purely because of their title or certification.
Recommendation 7: Leverage technologist leaders to define milestones, deliverables, timelines, risks and issues and consider using a project manager, or some project manager type of role, to simply organize and facilitate work measured against business objectives
Find a software model that expects, enables and equips capable software/hardware technologists to self-manage and self-evolve the actual decomposition and delivery of planned work. The creation of a work queue and prioritization of said queue should clearly and continue to lay in the hands of the corporation. Furthermore, the preparation of queued and prioritized work should continue to lay in the hands of self-encapsulated teams, primarily including business representation. However, rather than inserting a 3rd party project manager into the process of micro-managing/managing technologists who know their job better than the project manager, consider finding/developing a model whereby the technologist leaders themselves are expected to observe, define, decompose, size and deliver their work and potentially eliminate the need for a 3rd party intercessor.
Given we already discussed the need for predictable, repeatable delivery patterns built upon and measured by the ability to functionally and tangibly meet a business objective, expect the senior team/technologist lead to facilitate the decomposition and delivery of work within this delivery pattern, according to the business objective. Were we to arrive at this model, we could logically leverage a project manager in other roles within the organization with it may make more sense such as coordinating implementation, etc.
More work must be done in this area and is already underway as of this writing.