Wanderer LLC

I have a three recent examples of Extreme Programming failure, specifically “Collective Code Ownership”. Some of these examples are serious.

Collective Code Ownership means that everyone is responsible for all the code. Any developer may change any part of the code. An advantage claimed for collective ownership is that it speeds up the development process because if an error occurs in the code any programmer may fix it*.

Thankfully, we’ve abandoned most of the Extreme Programming Principles, but some of the attitudes persist. One example is our attitude towards developer resources. Except for a few key developers, it’s considered “boring” to leave someone working on the same codebase. Teams are often in firefight mode and need help. So, we shuffle developer resources from project to project, especially with the MACH Team, who often play a “sweeper” role.

The effect of this practice is that context, the important background and “whys”, are lost to the followup team being shuffled into a new project. As a result, the implementation isn’t satisfactory and the original team is perplexed with the follow-up team’s implementation, the code gets messier, or, worse, needs to be rewritten. A team that has to pick up where another team left off does not understand the decisions that were made, the history of the project, and the challenges.

Even when team leaders recognize the need to communicate the requisite context, the background description is often so complex, or communicated so poorly, that it’s too hard to grasp. This is multiplied across oceans and languages. It’s analogous to learning a foreign language by reading a book rather than doing.

How can that particular problem be eliminated? Answer: Stop the high frequency of shuffling. “That project is too boring” is not an excuse. It’s up to the leader to make it fun and challenging. I don’t think that any of our products are boring. There are all sorts of fun opportunties. A good leader will find them.

Here are my examples:

OEM Setup – Here’s an example where the MACH Team got lucky. The team was prepared to write a bunch of new (but easy) code to support the OEM setup scenarios when Robert, a WAI developer on 2.0, pointed out that the new code was not needed because the HWSW SDK Web Update APIs supported local downloads. In retrospect, I wish I would have recognized this and pointed the team to Patricia, who wrote the webupdate APIs, but I don’t think I was deep enough in the trenches to see it. Thankfully, a chance review by Robert saved us.

ITP Macro Feature – The Macro feature team was successful in delivering the feature for ITP, but it could have gone more smoothly with less architectural violations and hacks. This project also featured a huge amount of background and context needed, like three weeks worth, before the team could even start real work! My two examples:

Dependencies on the IP codebase violated the NextGen rules and make the component less portable.

We had to hack the code to hide the “Macro command…” and “Quick Turn” commands in ITP. This is easily solvable with the current architecture, but the team was not aware. When it was discovered, it was far too late to turn back.

ENDA – This was a serious lack of communication that resulted in a recreation of an existing architecture. We basically paid a lot more for a less robust and flexible solution. Having the right people involved would have avoided the problem.

I can’t stop now because here’s a fourth example happening right now as I write this entry. A developer in Redmond, who has reviewed MACH’s proposal for OEM’s Auto Update, lacks the background context of the original Auto Update. So, questions are naturally asked. Three MACH developers have now spent nearly two hours (and counting) discussing how to respond to the questions. The MACH Team only recently learned the context themselves. So, you have one developer, twice removed from the original implementation, spinning up a six-hour conversation (three developers times two hours…and counting) to help explain the context. In this problem, the answer is simple: Firmly state that you are remaining cosistent with the original implementation and stick with your design.

I’m not stepping in and to suggesting them what to say. This is their opportunity to learn.

The perception: A Redmond development team in gets in to trouble and needs help with fixing bugs. The MACH SW Team has functioned as a bug fixing house before, so we follow with tradition and send the bugs across the ocean. The MACH Team needs to ramp up on a new area and manage a high level of communication overhead (mostly in email). The MACH Team spends much more time than a team in Redmond would to understand what is needed.

There are two cultural influences on this issue that I see:

Mindfulness of authority – The MACH Team is generally more receptive to requests coming from authority figures (the Redmond Team).

Fear of mistakes – The MACH Team is generally more worried about making mistakes than their counterparts are in the US.

The result: A huge amount of wasted time. When large emails come across the ocean describing a task, every single word is scrutinized. Even if not every word is understood, team members are often reluctant to ask for clarification. The problem continues: more confusing and log emails and more time wasted reading emails. Even small suggestions in emails can send the team out on a wild goose chase. A simple suggestion of “You might want to look at this solution” can often be interpreted as “You should understand everything about this option because it might be a solution”. The reason? The cultural influences listed above.

What’s the perception on the Redmond side? My guess: “The MACH SW Team is slow! How can it take 3-4 weeks to fix such a bug?” A bad reputation ensues. Who’s fault is it? I personally don’t blame a single group. I see MACH making efforts to understand through study and training. But, I do not see such efforts in Redmond. Is this their fault? No. So, let’s make people in Redmond aware that communication improvement is a two-way street.

True story: Two team members on the MACH HWSW Team spent over three weeks each on two bugs. My question: How much did this help the team? Was it worth the effort to send these bugs to MACH? How can having two guys occupied for over 6 weeks of man-time on two bugs be valuable in firefight mode?

Six weeks on two bugs is a gross abuse of Pareto’s principle. Enter fear of conviction. Which choice would you pick:

Use your past experience and understand that throwing a bug over the ocean could occupy a magnified amount of someone else’s time. Instead of doing that, you find creative ways to eliminate the problem. Maybe that unit test isn’t so well-conceived, so you take a chance (with good payoff chances) and bend the rule of “always have unit tests” and cut the unit test in favor of a manual test case. You save the remote team lots of time, but get no credit for it because there’s not tangible “time saving” bottom line to credit you with.

Throw the bug over the ocean content that you’re following the rules. It’s now someone else’s problem that they can’t get the bug done in time; you’ve absolved yourself of responsibility.

In our environment, rational folks pick #2 and I don’t blame them, but it’s ridiculously ineffiecient. I see it over here. What is our goal with the MACH Team? To absolve our worry and responsibility or to have a productive team. I know the answer is the latter. To achieve that, the engagement of the MACH SW Team must be improved.

Do you want to have a productive team at MACH? How do you accomplish that with any team? One thing that is necessary is eliminating distractions from your team members so they can concentrate on what they do best.

What if I were to tell you that one email that takes you 3 minutes to write will take one MACH employee 30 minutes to read? Would you believe me?

The communication overhead at MACH is higher than you may think. One of the challenges English is the number of cliches in the language. When we speak or write “stream of conciousness” emails, we insert all sorts of language that is difficult for non-native speakers to deciper. For example:

“Just wanted to give you a quick update on what I’m up to with regards to getting things rolling for the MACH short releases.”

Confusing phrases include:

“what I’m up to”

“with regards to”

“getting things rolling”

How do we help the MACH team improve? Training helps. In fact, the MACH Team is continually working to improve their English and Email skills. But, what about Redmond? Maybe we can start thinking of ways that Redmond can improve so that MACH is enabled to be as productive as possible. What I’ve noticed is we always think of improving MACH. But, why can’t Redmond improve too?

In this case, you may think that because we’re native English speakers, we don’t need to improve our communication skills. That assumption is far from the truth. What if we at Redmond, learned how to write emails more succinctly, more clearly, without extraneous information and cliches? Wouldn’t we make big strides in reducing communication overhead?

I propose everyone in Redmond takes on a self-critical attitude of improving communicating between MACH and Redmond. Here are some ways we can start (note: this is what some form of training would address):

First of all, take the previous example:

“Just wanted to give you a quick update on what I’m up to with regards to getting things rolling for the MACH short releases.”

Remove the cliches and improve the sentence from a stream of conciousness to a cohesive sentence:

“Here is an update on starting MACH short releases.”

The first one sounds more friendly to native English speakers, but it requires a lot more reading time for people at MACH than the second. Also, the sense of tone from word choice is frequently lost when speaking in a non-native language. So, simplifying sentences becomes an even more powerful tool at MACH.

Also, be very careful with punctuation. In English, punctuation can change the meaning of a sentence and only a sophisticated user of the language can detect what you really mean. For example:

“For attended install UI will be displayed…”

What is the subject of this sentence? Is it:

1. “attended install UI”? In which case, the word “will” has accidentally been used in place of “to”.
2. “install UI”? If so, there’s a comma missing after “attended”.
3. “UI”? If so, there’s a comma missing after “install”.

In this case, the correct answer is “3″, but why give the pop quiz? Proper punctuation will make the sentence easy to understand.

The skill of clear written communication isn’t easy and just because you’re a native speaker does not mean you posess that gift. In most cases, it must be learned. A technical writing class may be helpful. Here are some of the things I try to do for my MACH audience:

When I think I’m done with an email, I go back through it one more time and:

Cut, cut, and cut. I’m always surprised at how much superfluous information I have. I try to distill my message into the shortest, most concise message possible. Less words means less time to communicate.

Simplify my sentences. Make them shorter, simplify the words. Big words aren’t needed, but the right words are.

Get rid of the cliches. English is full of cliches and it can be difficult for most native English speakers to communicate without them. However, it’s more difficult for non-native speakers to communicate with them! I maximize my chances of being understood by eliminating cliches.

Don’t communicate anything but the facts unless you have a good reason. I realize that my emails can get big if I try to communicate the “why”. Sometimes, this is important, but, believe it or not, most of the time it isn’t. I keep the “why” out unless it will help clarify someone else’s understanding.

How does one go about bringing changes to a group? Here are some thoughts about ways to do it when one does not have direct authority to do so. In some important cases, team leaders are what you need.

For simple changes, such as introducing a new tool, it’s sufficient for an influencer to simply show how a new tool will make work easier for team mates. If a tool doesn’t require effort to adapt to, has a small learning curve, and noticeably improves team member’s productivity, then it’s a fairly easy sell.

Other types of change, specifically cultural changes, are next to impossible without more engaged support from team leaders up the chain. These sorts of changes occur over a long period of time and need several small steps or requirements to be met in order for them to be successful:

The change should be easy to understand so that everyone knows what it is and can explain why it matters.

Demonstrate in clear terms a vision of how a change is useful and how it makes work lives better, and so on. Be able to communicate this clearly and quickly over and over.

Demonstrate successes of the change on a smaller scale. If the successes affect the bigger group, even better.

By this point, have buy-in from team leaders. Have them mention the successes at key opportunities as examples of best practices. Talk about vision. The goal is for the extended team to see it as valuable, making their lives easier, and, best of all, fun.

Eventually the change has to be championed by leaders on the team as an official best practice. Don’t stop talking about the vision.

Breathe a sigh of relief when the change becomes engrained in the team…months or years later.

For new team members, it’s up to that person’s manager and mentor to guide them to help integrate into the team by ensuring that their code meets the appropriate level of quality and that they’re aware of team norms and standards (like following the coding style guides of the code that you’re in).

Once that team member is meeting the expectations of the manager and the mentor, it’s expected that that team member be given more freedom to decide if he still needs code reviews from the manager or mentor. The manager or mentor may need to have the new member have code reviews from team mates instead.

It is always up to the project lead to also make the decision if code reviews are needed. These are above and beyond the training code reviews for a new member because the project lead (or a delegate) will be looking for project specific issues, adherence to specific guidelines, and so on. If the team member is not up to the expectation of the project manager, they should let appropriate manager know and the manager then decides the action to be taken. This normally isn’t a major failure. In most cases, it’s an example of misplaced expectations.

Those are the general rules. Of course, there are exceptions when the new team member, manager, and mentor needs to take different action. For example, a highly visible feature to important people would warrant extra scrutiny from the manager. etc, etc.

Here’s a great example of everyday mentoring at MACH. This was an opportunity to address two issues:

Thinking outside the box, getting creative with solutions to problems.

Finding information (this resulted in a TTT series).

The problem: When I saw how drawn-out the process was for fixing Lux bug 9778 (multiple emails, huge amounts of time for such a corner-case bug), I thought it might be a good opportunity to think outside the box and find an expedient solution.

My primary intention was not to solve the problem for Lux, but to help teach the team to recognize opportunities to find better solutions. In this case, the Lux developers were dead set on a particular solution that involved editing multiple files, dealing with elevated and non-elevated processes for copying and reading files, and lots of communication overhead. I used this opportunity to show Dante, the developer assigned this bug, that this was a good opportunity to look for a creative solution that avoided these headaches and hacks.

In this case, I gave him the idea: Why not print the file directly from the rich edit control? (See the bug for more.) He liked it and asked me how to do it. That was when I suggested that he search for the answer (the real purpose of this mentoring). I told him I would search for the answer too (I didn’t know if it was possible or easy) and we would compare notes after lunch.

I was able to find a solution in about five minutes.

After lunch, we compared notes and I was a little disappointed that Danted did not have an answer (he did find an alternate potential solution later on). I thought to myself, “Why?” It could be that he was not skilled enough in using search engine and other websites to find the information he needed. Hence, I decided to create the “Software Detective” TTT. I’m not convinced that this was entirely the reason for him not finding the right info. (In retrospect, I wished I would have asked him how he searched, but I didn’t.)

At the TTT, in addition to sharing some search tips, MSDN tips, and online source code depot tips with the team, I learned that if you want to find solutions to programming problems…

You had better know English (there’s only a puddles worth of information in Chinese compared to English’s oceans).

You need to understand the basics of how search engine queries work. For the most part, the team was already pretty good at this, but when they did run into trouble, they would uniformly run into trouble.

In the end, the Lux team did not accept this solution only because they were late in the cycle and this change would involve other teams’ buyoff. That didn’t matter to me; I was only interested in the team, Dante specifically, seeing and grasping the notion of backing up and trying something different.

This whole exercise was also good supporting evidence for a need for the Redmond team to randomize MACH less through more defined and planned projects, impovement of their own processes, and clear expectations of what the MACH Team should be used for and how they should be used.

The idea of the “TTT”, or “Tech Talk Tuesday” / “Tech Talk Thursday”, was my first big initiative for the MACH HWSW Team. The original intent was to simply improve the technical competency of the team through a series of bi-weekly technical presentations.

After assessing and getting feedback on gaps in technical expertise, I created a simple regimen for training to fill in the gaps. The majority of the training focused on “going back to the basics” and becoming experts in the most fundamental parts of software design and programming. Here are four TTT summaries of “Function Design” series:

• SDK Function Design
• Function Design 1
• Function Design 2
• Function Design 3

Preparing two topics every week was a lot of work. I then had the idea to involve the MACH SW Team to create presentations themselves. I still did the majority, but having three (then four, then five) team members collaborate freed up a lot of time. On top of that, asking for presentations from everyone on the team improved each team member’s competence in the following areas:

Knowledge of a technical subject. After all, this is an excellent way to learn a subject because you’re accountable to your team members. No one wants to get up and talk about something they aren’t well-prepared for.

Organization. Team members eventually learned how to best organize for an hour long technical discussion.

Confidence. With each successive presentation, each team member becomes more confident in their ability to communicate.

Communication. Teaching is a great way to improve communication skills.

The team was enthusiastic about presenting topics but everyone had a similar problem: organization. The problem manifested itself in presentations that were too broad. The presentations also went too long, often by twice as long or more (and we’d have to have multiple sessions). Eventually, each team member learned how to focus a topic and present in a simple and engaging manner and leave enough time for random questions. The more presentations a team member gave, the better they were at trimming the topic to fit the allotted time.

As of March 2008, they have exceeded my expectations.

We have had guest TTT speakers from time to time. For example, Leon gave us a great presentation on the basics of firmware.

Later, I became more experimental in my material. After noticing that some team members struggled to simply find information, I created a three part series called “Software Detective”, which focused on:

• Investigation – How to find information through websites, MSDN, and other resources.
• Intelligence – Tricks to using Spy++.
• Interrogation – Improving Debugging Skills

These sorts of TTTs, especially “Investigation”, contrast with super technical topics like “References and Pointers” which is designed to expose and inspire curiosity of how C++ works. This sort of understanding is again, in my opinion, fundamental to understanding how more high-level technology works.

What we have not explored in depth yet: MS Hardware SW application architecture. As of early March 2008, we have touched only on the NextGen component architecture in IP/ITP.

Overall, this has been successful initiative and the team, one year later, still looks forward to TTTs.

The “Side Projects” initiative encourages all team members of the MACH HWSW Team to always have a project that they are working on in their spare time. This spare time can be found in between feature development, after a project release, or simply a half hour every day at the end of the day. The side project must always have a tangible deliverable.

The idea of “Side Projects” came to me when I noticed that the MACH Team seems to expect almost all of their learning to come from training. “Training” is most often associated with TTTs. From my own experience, self-learning was critical to my development. I wanted to foster this attitude in the MACH Team. The more I thought about it, the more I liked the idea of “Side Projects”. I proposed the idea to Judy and she brought it to the extended team.

Here are the benefits:

Encourage independent learning. Training is important, but we already have an excellent training regimen in place and we can learn more quickly by working on side projects. Working on a side project on your own time requires managing time, independence, and individual decision making. Side projects accelerate learning. Doing produces tangible results.

Improve the team. The “team” can mean the MACH HWSW Team or the entire HWSW team. When a side project is completed, we have something that we can use, something that we can show to the team to demonstrate leadership and progress. We have a deliverable.

“Side Projects” is an important part of MACH HWSW’s overall development and investment.

Here are the side projects that are in progress at MACH as of this writing:

Redesigning cmdtest.exe to support NextGen components. This valuable test tool isn’t as useful anymore because it only supports dpgcmd.dll. Dante is working on improving the tool so that it can detect and test NextGen components like Magnify, Flip3D, etc. This project teaches user interface design and the NextGen architecture. This tool will benefit the DEG and Test teams.

Updating projomatic.exe. This tool has needed an update for a while. When the ICE team has stated the need as well, we decided to take action. Michael has taken this side project which will involve coordinating the updating of project settings that may have evolved (like how we handle exceptions, /Es, etc.), update the template .vcproj files to include builds for Prefast and 64-bit, update template .vcproj files to use the HWSW SDK. He will also have the opportunity to learn more about user interface design. This tool will be useful to the entire development organization.

Live Meeting. In order to improve communication between Shenzhen and Redmond, Marshall and Judy have taken on responsibility for training the MACH HWSW Team how to use Live Meeting. This will also encourage us to use our camera products every day. We will then plan to invite the Redmond team to use Live Meeting to communicate with us.

HID Client re-design. Marshall has taken on the task building on what Alex taught the team and prototyping ways to improve the HIDClient design. This is not an official rewrite, but an investigation on what is possible. This will teach design skills and a myriad of technical skills.

DEG User CPL Interface Improvement Prototypes. No owner yet. Our group has talked about this forever, but we’ve never had the time to seriously invest in it. We would like to investigate ways to improve the user interface such as moving to a hyperlink control model. Note: MHIX is a project in Redmond designed to solve this, so if someone is interested in this project, this could be a good opportunity for cross group collaboration.

DEG Reassignment Interface Improvements. No owner yet. Access to the reassignment user interface in DEG is terrible and badly out of date. We would like to investigate ways to quickly access the ability to reassign any button. Some ideas include following a My Favorites model, investigating double-clicks, investigating unobtrusive windows to remind users how to reassign buttons.

Device Detection. No owner yet. When we install our Mouse and Keyboard software, we still have to turn over our device in order to pick the correct device. We would like to investigate ways to improve this.

C# and DEG. Leverage learning from the ICE team to investigate WPF UI possibilities and the communication layer between the UI and the engine.

Expanded NextGen Component Design in DEG. Investigate Device Components (this overlaps with ENDA unfortunately) and Event Components. Also, write a demo to show the power of fully compartmentalized Command Components.

Here are the side projects that have been completed:

Simplifying the Unit Test System. This was very important for us because it shows that we can take the initiative and deliver a solution on our own. The test team did not have the resources or time, so Michael stepped up, analyzed the problem, arrived at a much simpler solution, and got it done! Other teams can learn from what we’ve done. They can also use what we have done. Note: We are still waiting on the Test team to plug it in; but we’ve gone well beyond what is normally expected of us.

Sharepoint Team Services. I created and customized a MACH HWSW Sharepoint which is be used by the team to manage common documents, research projects, and training materials. Much like coresw, this site is used to encourage collecting knowledge and ideas. We also have a useful searching engine working for us.