In 2015 and 2016 I was an adjunct instructor at the Startup Institute where I taught a half day class on Big Data. Now, half a day is not sufficient for a technical deep dive, so my aims were somewhat broader
Shortly after I first started working at a startup, I read the book How Google Works. A very readable summary of how Google operates as a company, how they are structured, and their culture.
One part of the book stood out to me, and changed how I approached my work was this:
For a meritocracy to work, it needs to engender a culture where there is an “obligation to dissent”. If someone thinks there is something wrong with an idea, they must raise that concern. If they don’t, and if the subpar idea wins the day, then they are culpable.
The word ‘obligation’ is important here. Personally I tend to avoid confrontation, and in my old line of work would need to be cajoled pretty heavily into participating in big group discussions, but I did take this advice on board, even though it was initially something that I had to force myself to do, heart pounding. I have to say that having this obligation has made me a much better engineer, and by not allowing myself to check out of meetings that aren’t directly relevant to my own immediate interests has made me more well rounded.
The obligation to dissent is especially critical in a small company where, say, five engineers may be responsible for the design and implementation of entire products. It is absolutely the responsibility of every engineer to understand, at least at a high level, the decisions that are being made. You can’t be an expert on everything, but you should at least know enough to be able to critically analyze and discuss what your peers are saying. The goals of this half-day session are:
- Survey the landscape of Big Data, and show the basic concepts and tradeoffs that need to be made. Do I need to be accurate? or fast? Do I need to be consistent? or accessible?
- Understand how the rising volumes of data are going to affect everybody. Focus on what we will all need to do to keep running in the face of ever growing data requirements.
I occasionally teach a two-day workshop on SQL. Designed for people with no database experience, this course takes you from the building blocks of databases, through being able to confidently write complex SQL queries. I developed all of the course materials for this bootcamp (the student handouts are available here). The course is very interacting and students are writing SQL queries throughout the day. On the first day, we cover the following topics:
- Basic DB concepts (the client, the server, the table, the row, the column)
- A brief history of databases
- Getting data from your database (select)
- Creating a new table (create table)
- Aggregating data (group by and basic aggregation functions)
- Filtering groups (having)
- combining queries (subqueries)
And on the second day we cover the following:
- Linking tables together (inner join, left join, right join, outer join)
- Well designed databases (normal forms)
- Making databases faster (indexes)
- An introduction to data projects with SQL