What is Reddit's opinion of Lean Analytics: Use Data to Build a Better Startup Faster (Lean (O'Reilly))?

From 3.5 billion Reddit comments

Without knowing too much, it sounds like you're supposed to make sure a hardware device gets set up so a user can use it. I could think a simple metric could be % of successful config completions. This would help you look across the funnel to see where they're falling out compared to the % of success.

If config is really the purpose, then it sounds like you don't really care about engagement or return metrics, just making sure they can use their purchase w/o friction. Return usage, in this case, could actually be a negative metric, like maybe the device is failing.

I started as a PM a few years ago and struggled with metrics getting started (still do sometimes). A book that I really like, which is a little older but still great, is Lean Analytics. It's marketed for startups, but it is a great primer for anyone in product or marketing looking to improve their use of analytics to achieve better outcomes. One thing they espouse in the book is that ratios are better for metrics since it helps ground it in comparables and take away some of the cruft from vanity metrics. Also, less is more. One great metric is always better than a few good metrics IMO.

Good luck and happy holidays!

These are all interesting metrics.
But it really depends on the type of your business and the goals you have with your website/app.

​

I highly recommend the book Lean Analytics that is all about finding metrics that matter most for your business and then focussing on them.

Focussing on metrics is a key part of a growth PMs role and I would absolutely recommend reading Lean Analytics https://www.amazon.com/Lean-Analytics-Better-Startup-Faster/dp/1449335675

I'd highly recommend reading the book Lean Analytics. It answers questions like yours at fundamental level for different business types. Here's the link - https://www.amazon.com/Lean-Analytics-Better-Startup-Faster/dp/1449335675

I highly recommend reading the book Lean Analytics https://www.amazon.com/Lean-Analytics-Better-Startup-Faster/dp/1449335675

This is a great video for some inspiration and frameworks https://www.youtube.com/watch?v=MABmQhOlmJA

https://www.productmanagementexercises.com/ has a great collection of questions and answers, though you might have to pay to access the entire question repository.

https://igotanoffer.com/blogs/product-manager/estimation-interview-questions might be helpful.

https://www.youtube.com/channel/UCjm_qVkCPjOVDz9BWjNqO9A (Exponent) and https://www.youtube.com/user/rocketblocks (RocketBlocks) have some good content on their youtube channel.

​

Hope this is useful :)

TLRD: I wanted to post this as a text post but I don't have enough karma. Posting here for now. If people find this useful, I'd love to move the disc to a self-post for other people to find this information more easily. I'm only posting part of it due to character limit

Summary: People in my personal life have asked for insight on breaking into the data science field/the interview loop. The following is a poorly formatted/continually updated list of my thoughts that I continually send out to people who've asked for them. I've decided to share it with the wider community. Apologizes for the poor formatting, I originally wrote this in email and I did not have the time to get the markup pretty.

Audience: People who are trying to break into data science and need help with the interview/job search. Early-mid career people might find some nuggets useful.

About me: Did my PhD doing experimental stuff with semiconductors. I'm comfortable with math and reading research papers, I'm a shit programmer. After grad school, I spent 2 years working for a no-name ML startup doing basic ML (mostly cleaning data, pipelines, feature engr experiments). I'm now a DS at FAANG-MULA for about a year. Opinions are my own, please feel free to disagree in the comments.

===================== CONTENT =====================

1. If you can code, consider looking into positions as a software engr. They make more money and there are about 10x more jobs than data scientists. The interviews at the lower levels are basically optimizing code that you can cram for via leetcode.com.

   1. Look up leetcode for programming problems. You should be able to solve most of the easy ones in ~3 minutes (warm up) and discuss big O, etc. Medium ones in ~7 minutes.
   2. Know SQL (joins, aggregations, and window functions) down cold. Keep in mind that SQL/pipelines often power plots in dashboards. This means all the business logic/transformations are done in SQL and the dash just visualizes it. You should be able to take raw data and format it into common figures (line chart, bart chart, histogram, etc). The most annoying part, for me, was remembering the different date functions (e.g. convert XYZ date format to quarterly date for aggregation). These tend to vary among different SQL dialects. Good companies won't get that you get the exact syntax of the function right. Also, look up fct and dim tables. I hate subqueries and I love CTEs. The easier you make it for your interviewer to read what you are doing, the better.
   3. Youtube lectures on ML I enjoyed. He also has course notes and what not somewhere on the internet. You may find other lecture series better and the curriculum is pretty standard at this level so don't feel attached to this one because I liked it. For DS roles that blend into MLE roles, you'll probably be asked to code some basic ML model. Linear regression, KNN, K-means, decision tree(s), etc. I've found engrs with more traditional CS backgrounds have some belief that their question digs at the heart of ML and that it's an effective screen. All will say that hiring is a noisy process. Maybe 1/3 will actually take steps to counter it. I've never seen anyone ask about SVMs though. I've even seen one company that asked people to code a Markov Chain in the 45 minute interview section. You'll almost certainly be asked how to make these methods scalable; you may or may not be asked to code the scalable method up in the short time frame.
   4. Some company tech blogs that could be useful:
   5. Instacart, in particular this one is a very good discussion on how to do a proper test. You won't be expected to be a master statistician, but you need to be able to show that your model/decision is better than the prior setting.
   6. The above blog referenced by Instacart is called a switchback experiment. DoorDash has some very detailed posts about it [1], [2], [3]. The details are not relevant for the interview, and I generally wouldn't expect a new DS to be familiar with this type of experiment in detail, but the general idea is worth digesting and it is interesting to see what a multi-year experimentation project could look like. Any company that has to deal with time AND location sensitive confounders will probably implement some version of this experiment.
   7. Lyft is also very good. In particular, this post (which focuses more on software engineering, but still very relevant) will give you a lot of insight on the other side of the table and what the interviewer is looking for.
   8. something to keep in mind in terms of having empathy for the hiring team: it likely costs ~1/2 million dollars/year to employ you. Your salary is ~200K. But once you factor in healthcare, payroll taxes, infrastructure (SV real estate ain't cheap), etc you've effectively doubled the cost to the company. That means you need to bring in ~1 million dollars/year in value. Also consider that new hires take 2-6 months to ramp, so that value delivery is backloaded. At the end of all your projects (and interview problems), you should be asking "Have I delivered enough value to justify my disgusting compensation package?"
   9. Also consider this Lyft post (contrast the decisions vs. algorithms data scientists) and this Airbnb post to see how data science often fits into the bigger picture. This airbnb post also talks about the different DS tracks.
   10. This post from DoorDash talks a little bit about their interviews and wanted business/communication sense. It is worth looking into combining MECE and funnel analysis to really structure your thoughts. Again, the point of interviews is not to answer the question, it is to show you approach the problem in a systematic way. If you can combine the two principles above, you can realistically list "all" the possible solutions. After that, the question is just how to prioritize which likely areas to investigate.
   11. DoorDash has a pretty heavy duty engr focus interview prep post, that likely isn't relevant to people pursuing a DS role but would be fair game for people looking to be an ML engr.
   12. Last point about the tracks, consider this post on metrics at Airbnb. It's a pretty stats heavy subject (even if the post is not super deep) - look at the author. She was a professor in statistics prior to Airbnb. Keep in mind what the competition looks like. It is worth noting my information applies to all the tracks. Some tracks may not ask you certain types of problems. For example, there may be tons of product/statistics types DS positions that would never ask you to write engr quality code.
   13. Another point about companies. It is worth realizing that many of the companies in tech (and the ones in this section) are marketplace companies.That means they create value by connecting buyers <=> sellers (and maybe shoppers and/or advertisers). That means these marketplace all deal with the same kinds of problems on both the business and technical side. An example of a market place post from Lyft.
   14. I really enjoyed the book Lean Analytics for a comparison of different tech company types and the metrics they should care about. And it has a good discussion about metrics in general. You should be able to find a pdf copy on library genesis.
   15. Taking all of the above, you really should expect a few types of product questions in your interview loop:

(a) Metric XX is going down. How would you investigate it? I always think about these problems from MECE + funnel analysis perspective as noted above.

(b) After expt AA, metric XX is going up but metric YY is going down. How would you think about it? This is a common problem where you're trying to understand tradeoffs/ambiguity and communication with managers/top line goals. If you EVER find yourself saying something definitive to this kind of problem, you're doing something wrong. Look up Pareto Frontier, but don't force it in.

[cut off due to character limit]

Since I see good suggestions for courses, some other things that data analysts use a lot

1. SQL - for data pulling and manipulation - you can learn it here interactively - https://mode.com/sql-tutorial/

2. Analytics concepts - Lean Analytics is a good summary of the methods needed to apply data to business problems - https://www.amazon.ca/Lean-Analytics-Better-Startup-Faster/dp/1449335675

3. For a overview of statistical concepts from a non-technical perspective, the recently released "Art of Statistics" is good to read even if you do more technical courses to help you with the bigger picture - https://www.amazon.ca/Art-Statistics-How-Learn-Data/dp/1541618513/ref=sr_1_1?dchild=1&amp;keywords=art+of+statistics&amp;qid=1606660024&amp;s=books&amp;sr=1-1

Lean Analytics: Use Data to Build a Better Startup Faster