Case Study – Professional Services ( PSO ) and Core Development ( R & D ) working together – lessons learned

Project: Outlook Add-in
PSO Devs code the the Add-in.
Core Development QAs test it.

Supported platforms:

  • Win 7 32/64 + Outlook 2007 / Outlook 2010 32/64
  • Win 8 Pro 32 / 64 + Outlook 2007 / Outlook 2010 32/64
  • Win 8.1 Pro 32 / 64 + Outlook 2010 32/64
    Service Packs available (if any)

Requirements:

  • Add-in must provide direct access from Outlook to resources located on a server.
  • Add-in must provide functionality 1:1 to the one available by accessing those resources through a web browser.
  • Add-in should not break any Outlook functionality or the one provided by other Add-ins.

People involved:

  • PSO developers – 1 to 4 (on and off the project).
  • PSO managers – 1 to 2.
  • PSO QAs – 1 – (on and off the project).
  • Core Development QAs – 1 to 4.

Professional Services

  • Focus on rapid development, many small to medium sized projects (single project takes under 4 weeks to complete).
  • Waterfall, no planning, short testing iterations.
  • Requirements always clear and provided by the customer.
  • QA engineers have supportive role – rarely consulted on how the customization/feature should behave – all is already written and signed by the customer.
  • QA engineers and Developers are taken in and out of projects according to current demand.
  • Developers (usually) run quick sanity checks of the code.
  • No dedicated QA manager – Department director is the QA manager.

Core Development

  • Focus on core product development – long release cycles – two releases per year – one major and one maintenance release.
  • Scurm / Agile, planning every 3/4/5 weeks, long testing iterations.
  • Requirements are rarely clear.
  • QA engineers have leading role in defining how given feature should behave.
  • QA engineers and Developers focus on their corresponding tasks.
  • Developers test their code and ensure basic functionality requirements are met.
  • Dedicated QA manager.

Shared management:

  • PSO decided what resource to use and when.
  • Core Development decided what needs to be addressed / changed as fixes and functionalists.
  • Neither team had experience with developing and testing Outlook Add-ins.
  • Neither team had knowledge of the possible technological limitations that arise.

Release Date:

  • First release date was mid October.
  • Second release date was mid November.
  • Actual release date was mid December.

Estimates:

  • First estimate was that the project would take around 6-7 weeks + 30% buffer.
  • Buffer ran out after the second week.

Results:

  • ~ 445 Defects, of which 384 were successfully regressed and 61 were deferred, no Block/Major defects were left open.
  • Add-in was released in mid December.
  • Requirements were met at around 90%, no supported platforms were removed.
  • People from either team met, talked, interacted, worked and had fun.

Lessons Learned:

  • There were a lot of reintroduced defects due changing the technology used (expected given the circumstances).
  • No paper trail – the requirements for add-in were (only) verbally stated as having the same functionality which was available in the End User (Web) Interface. They were poorly understood. PSO did not explicitly ask for requirements and Core Development was not proactive to provide such anyway.
  • First estimates were completely wrong given the fact that the project was deemed as “Red Alert” – meaning that this was a project with which PSO had absolutely no experience. Also this was the biggest project PSO have ever completed. Core Development insisted on at least doubling the estimates while PSO insisted on keeping them as they were. This lead to postponing the release several times.
  • Culture clash (expected) – PSO Devs were used to having a leading role in what is implemented and how in terms of functionality and behavior, Core Development QAs were not entirely happy with that – fortunately and due the great people working for either team this was resolved quickly.
  • PSO developers were working overtime. That’s never good.
  • There was no single person entirely responsible for the release in either team – responsibilities were shared but that sharing was not a clear one and everyone was expecting the other party to act whenever necessary. Nobody acted and assumed full responsibility. Having one person responsible or clearly stated shared responsibilities is a must even if everyone have proven record of always meeting dead-lines and successfully leading projects.
  • Planning alone is not enough, sometimes you have to improvise.:)
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s