Random Musings on Technology

Evaluating for a Project Manager

 • 

I was asked to describe how to evaluate a Software Project Manager. I remembered this nice challenge I did at some point, and realized that had to dig in memory a lot (and ask someone who actually had taken the challenge). For that reason I decided to write about it here.

At some point in the past I have had to hire a Software Project Manager for a startup. We were looking for someone who had great Project management skills, but that also understood the Engineers.

As part of the evaluation process I did an "onsite challenge", which took around 3 hours. For this challenge the candidate had to define a plan (in the form of a type of Gantt Chart) for executing a Software development project, given a set of constraints (in time, resources, and scope of course).

The Challenge

The objective of the challenge was to test how the candidate handled the definition of a real-life project, with its constraints. The constraints came in the form of insufficient resources and time, for the scope

The candidate was recommended to ask questions to the interviewer, who played the roles of a) The Engineering team and b) The Product Team. The points to evaluate were:

  1. How well the candidate interacted with the Engineers and Product "teams" to ask them questions, clarification, etc.
  2. How much "domain knowledge" did the candidate have allowing him to make sound decisions when making compromises.
  3. How did he handled the scope/resources duality and what solutions did he came up with to address them.

Scope

To define the scope of the project, we provided the candidate with a list of stories needed to develop a mobile App. This list was comprised of Frontend stories (to implement standard UI HTML/JS), Backend stories (for the APIs to be used by the app) and Mobile stories (which required specialized mobile knowledge).

The stories were provided in the form of Story Name - Story Points - Skill, where Skill defined which type of Engineer could do it (everyone, Front End only, Backend only, Mobile Only). The stories were real stories (they actually were partially based in another project we had made before).

Resources

We provided a set of theoretical Engineers for the project. Engineers were defined in 2 dimensions: Seniority (Jr., Mid level, Sr, Tech Lead) and Skills (Fronted, Backend, Full Stack, Mobile). Each Engineer had a specific seniority and a combination of Skills they had.

The Seniority factor was modeled as the number of story points that an Engineer could finish per Week.

The idea here is that the Project Manager candidate had to consider the skills and seniority when assigning each story to the different Engineers.

Time

The last of the constraints was modeled as the date when Business had asked us to release the software product. The available time of course was not enough to implement the full list of requirements considering the available resources.

Moreover, as part of the definition of the problem, we asked the candidate to consider people going on vacation and getting sick (I barely remember even adding some vacation days for some of the Engineers, although I am not sure of that).

Evaluation

The desired output of the challenge was a type of Gantt chart. We did not explicitly asked for this, but asked the candidate to use any tool hey could, so that at the end of the challenge, he would be presenting to "Business" (i.e. the CEO, given that this was a small startup) the plan for realizing the project. The best candidates usually made some kind of Gantt chart.

The output was only one aspect of the evaluation, another (more important to me) was the interactions that happened while doing the challenge; these in the form of questions to the Development team, or questions to the Engineering team.

As candidates asked questions, possibilities opened here and there. For example, one story that was 8 points if he asked if it could be split in two, the answer was usually yes (given that those were real stories, I tried to stay true to the story and give a "real world" answer. For example if the 8 point story was a complex frontend screen in the App, divided it in 2 stories of 5 and 3, dividing the screen development in two parts).

At some point some candidates even asked about QA considerations (something I did not consider initially), so we gave them some "advantage" (in the form of time, telling him that say, the stories had some QA in them, which could be split and maybe better accommodated).

The one thing that we were looking for was for candidates that not only filled up the scope until filling the time available in the project, but that went a bit further away.

My experience with startup projects has been that there is always not enough time to do all the required features, but the greatest value (and challenge) comes into prioritizing the tasks so that the team can deliver the mythical MVP on time, and then, perform additional iterations improving the product. This type of reasoning is what we were looking for as part of the output of the challenge.

Results From Applying The Challenge

In general, candidates that asked more questions produced a better result. We interviewed about 30 different people for the position and we had a very diverse set of responses.

Some candidates had a stronger technical background, and thus could leverage more the content of the stories. Other candidates had not so much technical background and leveraged more the business side.

In general, the challenge was a success, and in the end I remember we found the right candidate for the team.

Something important to stress is that, this like many other "technical" (in the sense of, having to implement something, in this case a plan) interviews tends to suffer from a "degradation of context" problem: When we created it, it was very clear for us that the evaluation consisted in not only the end product (gantt chart, or anything else) but the interaction and general process that happened during the 3 hours. If other people start using this challenge, it is important not to lose this context and only consider the end product for the scoring, as it will be misleading.

I've seen this happen in other types of challenges (specially ones that comprise programming something). Where inexperienced interviewers decide to apply the problem for an interview and score it as in a "black and white" given the program provided by the candidate.

The Whole Point of Blockchain

 • 

I loved this tidbit from this MIT technology review article describing the point of the Blockchain technology:

The whole point of using a blockchain is to let people—in particular, people who don’t trust one another—share valuable data in a secure, tamperproof way. That’s because blockchains store data using sophisticated math and innovative software rules that are extremely difficult for attackers to manipulate

Evaluating non Technical Employees

 • 

A good post on how on how to evaluate the non-technical side of employees. They used this framework to evaluate Product Design & Development collaborators:

https://mikeindustries.com/blog/archive/2016/06/evaluating-employees-in-product-design-development-roles

They used what they called the "four pillars". And evaluated by asking a person's peers to score them:

  • Getting things done: Pretty simple. Do you do what you say you are going to do? Do you go the extra mile to complete work that needs completing? Are you where problems go to die? This pillar is both a measure of dependability and prolificacy.

  • Creating strong relationships: Product development is a team sport. As I mentioned in Three Years in San Francisco, Intelligence X Collaboration = Results. Note that the relationship is multiplicative. If either I or C nears zero, so does R. Unless you are running the world’s worst interview process or physically sucking brain matter out of your co-workers, I is never really in danger of hitting zero. C, however, always is. If you score highly in this pillar, not just your direct colleagues will love working with you, but your cross-functional colleagues especially will. Know that designer who all engineers want to work with? That’s this woman.

  • Improving the team: One of the benefits of working at a company with other great designers, engineers, PMs, and researchers is that the sum can be a lot greater than the parts. Not only that but by virtue of being around so many talented teammates, your own career growth can happen almost automatically without having to spend nights and weekends learning new skills. We encourage people to make their teammates better by doing things like brown bag sessions on prototyping, brainstorming, and other important skills. Additionally, we encourage people to proactively help their teammates out on projects even if the project is not their personal responsibility. Improving the team can come in many other ways, including recruiting new teammates, but the point is to be a great teammate above and beyond being a great designer, engineer, etc.
  • Technical skills, empathy, and vision: These are the individual skills that most people initially assume are the only keys to success and promotion. We purposefully made them account for only 25% of the total formula to stress how important the other elements are. These are also the skills that are most customized to Design & Research. If you want to adapt this entire framework to PM, Eng, or any other function, you could probably leave the first three alone and just change this one pillar a bit.

The Principles of Good Programming

 • 

I have always liked this article that describes some principles of good programming. I count is as part of the knowledge that differentiates a coder from a Software Engineer (like, doing real Engineering):

https://www.artima.com/weblogs/viewpost.jsp?thread=331531

(The following text is taken verbatim from that page)


This is the kind of stuff that you are not taught in school and will definitely differentiate your from a Jr or even Mid level developer.

The principles of good programming are closely related to principles of good design and engineering. The following programming principles have helped me over the years become a better programmer, and I believe can help any developer become more efficient and to produce code which is easier to maintain and that has fewer defects.

DRY - Don’t repeat yourself - This is probably the single most fundamental tenet in programming is to avoid repetition. Many programming constructs exist solely for that purpose (e.g. loops, functions, classes, and more). As soon as you start repeating yourself (e.g. a long expression, a series of statements, same concept) create a new abstraction. http://en.wikipedia.org/wiki/Don%27trepeatyourself

Abstraction Principle - Related to DRY is the abstraction principle “Each significant piece of functionality in a program should be implemented in just one place in the source code.” http://en.wikipedia.org/wiki/Abstractionprinciple(programming)

KISS (Keep it simple, stupid!) - Simplicity (and avoiding complexity) should always be a key goal. Simple code takes less time to write, has fewer bugs, and is easier to modify. http://en.wikipedia.org/wiki/KISS_principle

Avoid Creating a YAGNI (You aren’t going to need it) - You should try not to add functionality until you need it. http://en.wikipedia.org/wiki/YAGNI

Do the simplest thing that could possibly work - A good question to ask one’s self when programming is “What is the simplest thing that could possibly work?” This helps keep us on the path towards simplicity in the design. http://c2.com/xp/DoTheSimplestThingThatCouldPossiblyWork.html

Don’t make me think - This is actually the title of a book by Steve Krug on web usability that is also relevant in programming. The point is that code should be easily read and understood with a minimum of effort required. If code requires too much thinking from an observer to understand, then it can probably stand to be simplified http://www.sensible.com/dmmt.html

Open/Closed Principle - Software entities (classes, modules, functions, etc.) should be open for extension, but closed for modification. In other words, don't write classes that people can modify, write classes that people can extend. http://en.wikipedia.org/wiki/OpenClosedPrinciple

Write Code for the Maintainer - Almost any code that is worth writing is worth maintaining in the future, either by you or by someone else. The future you who has to maintain code often remembers as much of the code, as a complete stranger, so you might as well always write for someone else. A memorable way to remember this is “Always code as if the person who ends up maintaining your code is a violent psychopath who knows where you live.” http://c2.com/cgi/wiki?CodeForTheMaintainer

Principle of least astonishment - The principle of least astonishment is usually referenced in regards to the user interface, but the same principle applies to written code. Code should surprise the reader as little as possible. The means following standard conventions, code should do what the comments and name suggest, and potentially surprising side effects should be avoided as much as possible. http://en.wikipedia.org/wiki/Principleofleast_astonishment

Single Responsibility Principle - A component of code (e.g. class or function) should perform a single well defined task. http://en.wikipedia.org/wiki/Singleresponsibilityprinciple

Minimize Coupling - Any section of code (code block, function, class, etc) should minimize the dependencies on other areas of code. This is achieved by using shared variables as little as possible. “Low coupling is often a sign of a well-structured computer system and a good design, and when combined with high cohesion, supports the general goals of high readability and maintainability” http://en.wikipedia.org/wiki/Coupling(computerprogramming)

Maximize Cohesion - Code that has similar functionality should be found within the same component. http://en.wikipedia.org/wiki/Cohesion(computerscience)

Hide Implementation Details - Hiding implementation details allows change to the implementation of a code component while minimally affecting any other modules that use that component. http://en.wikipedia.org/wiki/Information_Hiding

Law of Demeter - Code components should only communicate with their direct relations (e.g. classes that they inherit from, objects that they contain, objects passed by argument, etc.) http://en.wikipedia.org/wiki/LawofDemeter

Avoid Premature Optimization - Don’t even think about optimization unless your code is working, but slower than you want. Only then should you start thinking about optimizing, and then only with the aid of empirical data. "We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil" - Donald Knuth. http://en.wikipedia.org/wiki/Program_optimization

Code Reuse is Good - Not very pithy, but as good a principle as any other. Reusing code improves code reliability and decrease development time. http://en.wikipedia.org/wiki/Code_reuse

Separation of Concerns - Different areas of functionality should be managed by distinct and minimally overlapping modules of code. http://en.wikipedia.org/wiki/Separationofconcerns

Embrace Change - This is the subtitle of a book by Kent Beck, and is also considered a tenet of extreme programming and the agile methodology in general. Many other principles are based on the concept that you should expect and welcome change. In fact very old software engineering principles like minimizing coupling are related directly to the requirement of making code easier to change. Whether or not you are an extreme programming practitioner, this approach to writing code just makes sense. http://www.amazon.com/gp/product/0321278658