Harris Ossermanhttp://harrisosserman.postach.io/feed.xml2020-05-27T17:22:49.902000ZWerkzeugUser testing and user feedbackhttps://harrisosserman.postach.io/post/user-testing-and-user-feedback2020-05-27T17:22:49.902000Z2020-05-24T20:51:49ZHarris Osserman<div>Over the past two years, I’ve tried some relatively unconventional strategies to user test <a href="https://www.talkhiring.com/">Talk Hiring</a>. I’m proud to have worked on a product that is so easy to use that people who claim to be "bad at computers" can still use it. </div>
<div><br /></div>
<div><span style="font-weight: bold;">First off, a book.</span></div>
<div><br /></div>
<div>I recommend <a href="http://momtestbook.com/">The Mom Test</a> as a good primer on how to elicit feedback from users without biasing them. Or as the author describes it: "how to talk to customers & learn if your business is a good idea when everyone is lying to you." At the bottom of the website linked to above, you can enter your email and get some of the book content for free.</div>
<div><br /></div>
<div><span style="font-weight: bold; text-decoration: underline;">Face-to-face guerrilla testing strategies:</span></div>
<div><br /></div>
<div><span style="font-weight: bold;">Train stations.</span> I’m not talking about Penn Station, Grand Central, or the like. I would go to my local suburban train station that has 2 tracks. Commuters waiting at the train are as captive an audience as they come. Plus, with train schedules, you can figure out how much time they have to wait for. I would head over there when the trains to New York Penn Station and Hoboken would be stopping there within 15 minutes of each other, to maximize foot traffic. It was pretty easy to walk the platform and find people willing to try out a new product, many willing to do it for free. Once I moved to the Bronx, I did the same at the local Fordham train station.</div>
<div> </div>
<div><span style="font-weight: bold;">Hiring events.</span> Talk Hiring helps people improve their interviewing skills, so naturally, we found good user testing subjects at hiring events. Specifically, I wanted to target very busy hiring events where job seekers had to wait to talk to employers. Suburban job fairs are relatively empty compared to NYC job fairs, where the lines can easily snake around the block for the opportunity to talk to a handful of employers. I would walk the line and ask bystanders if they wanted to test out whatever it was that I was working on. People waiting on line at a job fair are feeling a combination of anxiety and boredom, and offering them the opportunity to try something out was welcomed by many. <img src="https://cdn-images.postach.io/d7832041-1d7e-4ec7-bb7e-099193cc8c7d/b43e5d3b-469f-483f-8397-9b46ea62b113/c851e70a-c255-423b-9ee2-b8c9d4487cb3.jpg" /></div>
<div><br /></div>
<div><span style="font-weight: bold;">Stopping people on the street.</span> This did not work well at all for me. Some may have more luck at it, but I found that people hate the intrusion because they’re always in the process of doing something or getting somewhere. In my neighborhood of the Bronx (Morris Park), I also found a lot of people didn’t speak English. The Bronx has a hispanic majority, but I do think some people used this as an excuse to avoid me.</div>
<div><br /></div>
<div><span style="font-weight: bold;">Paying people for participating in user tests. </span>Having a bunch of money in my back pocket seemed unsafe, so instead, I had an assortment of Amazon and Uber gift cards with different denominations in my backpack. In NYC, some of the older folks didn’t really use Amazon, but they did use Uber to get around in emergencies.</div>
<div><br /></div>
<div><span style="font-weight: bold;"><u>Online user testing strategies:</u></span></div>
<div><br /></div>
<div><span style="font-weight: bold;">Userbob (</span><a href="https://userbob.com/" style="font-weight: bold;">https://userbob.com/</a><span style="font-weight: bold;">)</span><span style="font-weight: bold;">. </span>If you have something online that people can test from their home, Userbob is the most affordable online user testing site I’ve found. They charge $1/minute, and they will recruit the study participants for you. If I wanted some quick feedback on something I was building, I could spend $25 and have 5 people spend 5 minutes testing it out within a day or so. It’s flexible to the minute in terms of time increments. They share screen and audio recordings as people go through the steps in the user test. </div>
<div><br /></div>
<div><span style="font-weight: bold;">Collecting user feedback:</span></div>
<div><br /></div>
<div><span style="font-weight: bold;">Quick Zoom screen share calls with users as they encounter problems.</span> A few months ago, we integrated live chat into Talk Hiring. When users reach out about a problem (and I have a free moment), I try to hop on to a Zoom call with the user ASAP. This has been incredibly helpful because I can see exactly what is confusing or broken to the user. I can ask follow up questions, and we can strategize which product changes could help future users. I’ve made numerous small changes to our product because of these quick Zoom screen share calls. I can learn a lot more from a screen share than from a live chat conversation. </div>
<div><br /></div>
<div><span style="font-weight: bold;">Weekly calls with pilot users.</span> When Talk Hiring was in the early stages, we ran pilots with job training programs. One of the requirements for running a pilot was that we had to have weekly phone calls with someone at the program to get quick and constant feedback on what was working and what wasn’t. I also used these calls to talk about the new bugs we fixed and features we shipped in the past week. Users love hearing when you build something that was their idea.</div>
<div><br /></div>
<div><span style="font-weight: bold;">User surveys.</span> After every mock interview, users can fill out an optional 3 question survey. We automatically tag the feedback as coming from a specific user and career readiness program. We’re then able to look for trends in our feedback across organizations and within organizations. This is pretty common stuff, so I won’t go into too much detail here. We do share the results of these surveys with our partner programs so that they know what their participants think of Talk Hiring (usually very positive opinions).</div>
<div><br /></div>
<div><br /></div>
How much should it cost?https://harrisosserman.postach.io/post/how-much-should-it-cost2020-05-18T19:35:20.983000Z2020-05-16T03:28:18ZHarris Osserman<div>Talk Hiring sells its mock interviewing tool primarily to nonprofit job training programs, and more recently, high schools. There are playbooks online on how to price online educational tools for schools, but not as much for nonprofits. I’ve spent many days squirming around, trying to figure out the optimal strategy for pricing the tool, and I thought that walking you through my pricing evolution may be helpful.</div>
<div><br /></div>
<div><span style="font-weight: bold; text-decoration: underline;">June 2019: No Longer Free Forever.</span></div>
<div>We offered custom flat-rate discounted pricing based on the number of potential users. New programs were offered 6-week free trials. <a href="https://harrisosserman.postach.io/post/transitioning-from-free-pilots-to-a-paid-product">Link to blog post</a>.</div>
<div><br /></div>
<div><span style="font-weight: bold; text-decoration: underline;">August 2019. No free trials</span><span style="text-decoration: underline;">.</span></div>
<div>Instead of free trials, we offered 60-day money back guarantees and continue to price based on the number of potential users. It was the early days of charging for Talk Hiring, and I wanted to work with only serious prospects with budget. For the programs that decided that they didn't like the product, they could effectively run a free trial with us with our money-back guarantee. </div>
<div><br /></div>
<div>Why we moved away from this pricing:</div><ul><li><div>Although we did bring on 2 paying customers with this pricing, it became obvious pretty quickly that this was not how nonprofits wanted to buy software. Finding money for anything in a nonprofit can be a battle, and asking for money before we had proven our value was a hard sell. </div></li><li><div>Larger nonprofit job training programs were not willing to agree. Contracts in the $5,000-$15,000 per year range were too significant financially for these programs to be comfortable essentially loaning us a big chunk of change for 60 days while they test out this product. For larger programs, I could have offered a more nominal per month cost when starting out. That may have helped a little, but we still would have been facing the obstacle of bullet #1.</div></li><li><div>Talk Hiring was (and still is) a new business and many nonprofits are risk-averse, rightfully so. A money-back guarantee is only good as long as Talk Hiring survives, and some potential customers were not 100% convinced that they would be able to get their money back. </div></li></ul><div><br /></div>
<div>Once Talk Hiring is more mature, I would like to revisit this pricing strategy. It works well when potential customers have a high-degree of confidence in your business and the price that they’re paying while testing it out is relatively low. </div>
<div><br /></div>
<div><span style="font-weight: bold; text-decoration: underline;">October 2019. Free Trials + Usage-Based Pricing with an Upper Ceiling</span><span style="text-decoration: underline;">.</span> </div>
<div>We had a price calculator on our website where programs could enter the exact size of their enrollment, and get monthly and annual price quotes for their program. We changed our pricing to offer $10 per mock interview, with an upper ceiling of $20 per user per month. The upper ceiling was to give customers the assurance that the cost would never be above (# of users) * $20 in a given month. </div>
<div><br /></div>
<div>Why we moved away from this pricing:</div><ul><li><div>Although this is a common way to price software, asking a job training program to know the exact number of people that would want to use a mock interviewing tool is next to impossible. Plus, many job training programs support low-income populations that are more transient in nature. </div></li><li><div>For budgeting purposes, nonprofits really want to know how much something is going to cost. Of course, they also want the best price that they can get, but all else being equal, flat rates are preferable. The price could swing by hundreds or thousands of dollars from month-to-month with usage-based pricing.</div></li><li><div>Usage based pricing works well when the price is very low (think AWS charging cents per hour of server time). We don’t want instructors thinking, "should I do another lesson with Talk Hiring this month or save the money for something else?"</div></li></ul><div><br /></div>
<div><span style="font-weight: bold; text-decoration: underline;">November/December 2019. Pay What You Want Pricing and Tiered Pricing.</span> </div>
<div>Pay What You Want: I read a book by <a href="https://tommorkes.com/pwywguide/">Tom Morkes on Pay What You Want</a> one Saturday morning and was intrigued. Nonprofits (especially religious institutions and museums) use this pricing strategy all the time. Plus, I was getting tired of trying to figure out how to price this tool, and thought, why not dare my customers to tell me how much they would pay for it. Given that my customers know me well, I thought that it would be awkward for them to decide to pay $0 or something only a little bit above $0. I also thought it could help generate good will amongst my customers.</div>
<div><br /></div>
<div>Tiered Pricing: I talked to as many financial professionals at nonprofits as I could (CFO, VP of Finance, etc) to learn how they like to purchase software. The overwhelming majority wanted a predictable, good price (with a nonprofit price discount). </div>
<div><br /></div>
<div>How I tested these two pricing strategies:</div><ul><li><div>A/B tested an email campaign to different prospects. Option A was our flat-rate, tiered pricing and Option B was pay what you want.</div></li><li><div>I reached out to a few recent prospects that I had talked to, and offered them pay what you want pricing. Pay what you want didn’t motivate any of them to move forward, which really surprised me. Of course, there are many reasons why they may not have moved forward, but given that these were prospects with real interest in the product, I was surprised.</div></li><li><div>I reached out to <a href="https://www.google.com/search?sxsrf=ALeKk03f4cMzaUa7w_njTirDjEcUALIP0g%3A1589688705006&ei=gLnAXtyBPfmxytMPhdeJoA8&q=tom+morkes+pay+what+you+want&oq=tom+morkes+pay&gs_lcp=CgZwc3ktYWIQAxgBMgQIIxAnMgQIIxAnOgIIADoGCAAQFhAeUOsOWOURYIUbaAFwAHgAgAFOiAGeApIBATSYAQCgAQGqAQdnd3Mtd2l6&sclient=psy-ab">Tom Morkes</a> for his advice on using pay what you want in a b2b scenario. He said that pay what you want works best when a single seller is working with a single buyer because it requires the human element to work. Selling to committees or groups removes that human element. </div></li></ul><div><br /></div>
<div>We still run tiered, flat-rate pricing to this day.</div>
<div><br /></div>
<div><span style="font-weight: bold; text-decoration: underline;">April 2019. Simplified, tiered pricing.</span></div>
<div>I came to a few realizations:</div><ul><li><div>We had too many pricing tiers. We had 7 different pricing bands based on the number of participants that a program worked with in a given year. Most of our customers were paying $100-$200 per month or $400-600 per month. We should simplify and charge the midpoints of our most popular pricing bands: $150 or $500 per month.</div></li><li><div>The number of staff that want access to our staff dashboards scales with the number of users in the program, for the most part. As I mentioned before, estimating enrollment in a job training program is much harder than estimating the number of staff that will be involved in prepping job seekers for interviews.</div></li><li><div>COVID-19 was directly impacting job training programs (skyrocketing unemployment paired with tenuous funding). By pricing based on staff dashboard access only, we are able to open our doors to as many job seekers as programs need to support during this time of crisis.</div></li><li><div>My friend <a href="https://www.linkedin.com/in/soroushjp?originalSubdomain=au">Soroush</a> shared <a href="https://capitalandgrowth.org/answers/Article/3169972/The-Definitive-Guide-to-Pricing-Plans">this article</a> with me on pricing best practices. I’ve read many articles on the topic, but I found this one especially concise and easy to implement. It talks about price anchoring, the decoy effect, and the paradox of choice, and I used all of these tactics in my new pricing.</div></li></ul><div><br /></div>
<div>As of writing this post, here is a screenshot of the Talk Hiring pricing page. </div>
<div><br /></div>
<div><img src="https://cdn-images.postach.io/d7832041-1d7e-4ec7-bb7e-099193cc8c7d/1bd4ad9a-a1f3-4277-9e65-91b18dc79962/c457b107-08b2-4c83-aa4d-dc9b1c8269ca.png" title="Tap to Download"/><br /></div>
<div>I’m sure in a few months, I’ll have a new take on our pricing, but for now, I’m happy with it!</div>
<div><br /></div>
<div><span style="font-weight: bold;"><u>Thoughts:</u></span></div>
<div>Flat-rate tiered pricing is the norm for selling SaaS products, but the real challenge is figuring out what the different price points and package features should be. I don’t regret the winding road that took me to this point. By experimenting with so many different pricing strategies, I was able to learn what works and what doesn’t work for Talk Hiring’s customers.</div>
Transitioning from free pilots to a paid producthttps://harrisosserman.postach.io/post/transitioning-from-free-pilots-to-a-paid-product2020-05-15T04:15:49.167000Z2020-05-14T04:08:31ZHarris Osserman<div>We launched our <a href="https://www.talkhiring.com/">mock interviewing tool</a> in late January of 2019. This initial product was pretty basic, but we had honest interest from a handful of job training programs. We ended up running about 10 free pilots over the course of 4-5 months. Transitioning from free pilots to a paid product was messy and awkward.</div>
<div><br /></div>
<div>Here’s a photo of me doing a pilot kickoff at <a href="https://northernnj.dressforsuccess.org/">Dress for Success Northern NJ</a>:</div>
<div><img src="https://cdn-images.postach.io/d7832041-1d7e-4ec7-bb7e-099193cc8c7d/6b5fc480-7384-4dc6-9013-108fcb16af58/5a6e68a3-e979-4202-a18b-a03cd43c6144.png" /></div>
<div><br /></div>
<div>I got into this business by thinking that voice is a much better interface than text for many job seekers. I still think that. My plan was to give out our automated mock interviewing tool for free, provide value to a lot of job seekers, and then bring on employers to start connecting them via an online recruitment marketplace. But within 4-5 months of running free pilots, I realized that my strategy likely wouldn’t work. Why?</div><ul><li><div>Convincing and then onboarding organizations to a new product is a slow process, and as a self-funded business, I needed to get to breakeven as quickly as possible. At the <a href="https://nextgenhq.com/">Next Gen Summit</a>, David Rose, a prominent angel investor, said that the top 5 rules of entrepreneurship are all the same: don’t run out of money. If we wouldn’t be paid until enough job seekers were using Talk Hiring for employers to be interested, we wouldn’t be able to even attempt to generate revenue for an additional 6+ months. By charging for the product now, I would be able to de-risk the financials while still leaving the future goal of working with employers on the table.</div></li><li><div>We were able to sign on some national programs that work with thousands of job seekers as pilot partners. But, every large program understandably wanted to pilot the tool with only a segment of their organization. Even if we ran successful pilots, our growth within organizations would be slow moving.</div></li><li><div>It was an awkward process for our pilot partners to run a free pilot for a free tool. We set up the pilot process really as a way to learn about how to make the product better in a controlled setting. But, it’s hard to really take a pilot seriously when, after the pilot, the pricing is $0. By charging something for Talk Hiring, partners take the free trial more seriously because they know that they’re going to have to gather strong evidence to convince the team to eventually buy this tool.</div></li></ul><div><br /></div>
<div><span style="font-weight: bold;">When did we transition from free pilots to a paid product?</span></div>
<div><br /></div>
<div>Once we had testimonials from happy users, a product that a few job training programs seemed to be getting value out of, and a <a href="https://docs.google.com/document/u/1/d/1sqLllYxWRr9UV6WQd86SKwz5zGVY26utCNLHx2mSqaI/edit">small study</a> to measure the effectiveness of the tool.</div>
<div><br /></div>
<div><b>Why was it awkward/messy?</b></div>
<div><b><br /></b></div>
<div>I thought that the Talk Hiring mock interviewing tool would be free forever, and all of our pilot partners did too. I had to communicate why we were changing our strategy with each pilot program, inform all upcoming fall pilots of this change, and figure out how to price the product for the innovators that took a chance on me and this brand new tool. About half of the programs that had really used the tool during the 3-month pilot program paid a (very discounted) price.</div>
<div><br /></div>
How Talk Hiring got its first usershttps://harrisosserman.postach.io/post/how-talk-hiring-got-its-first-users2020-05-14T21:11:02.833000Z2020-05-14T03:03:01ZHarris Osserman<div>Talk Hiring (<a href="https://www.talkhiring.com/">talkhiring.com</a>) is an automated mock interviewing tool. To date (May 2020), we’ve conducted over 3,000 automated mock interviews with job training programs, high schools, and colleges. I’m the Founder/CEO and have been working on this business full-time since August 2018.</div>
<div><br /></div>
<div>Talk Hiring started off as an automated phone screening tool for employers. We simply wanted to help employers save time and speed up their hiring process by automating their initial phone screens with job seekers. This business idea didn’t last long (<5 months).</div>
<div><br /></div>
<div>To find businesses that might find our tool useful, I started by pounding the pavement and walking into small businesses in NY/NJ. Although it’s free to do, I wasn’t very successful (however, this did connect me to Talk Hiring’s first employee, Anson Ervin). Some reasons:</div><ul><li><div>I hadn’t thought through the fact that I should have professional business cards. I was ripping off sections of paper from my notebook with my contact info. Those "business cards" had a response rate of 0%.</div></li><li><div>The manager of a store/restaurant is often not present (or maybe they were lying to me). </div></li><li><div>Most of these small businesses don’t do lots of hiring, so automating job interviews isn’t that exciting.</div></li><li><div>The second that a customer walks in, I was less important.</div></li><li><div>Nobody likes walk-in solicitors. I distinctly remember being yelled at more than once.</div></li></ul><div><br /></div>
<div style="text-align: center; ">A replica of my initial business cards :)</div>
<div><img src="https://cdn-images.postach.io/d7832041-1d7e-4ec7-bb7e-099193cc8c7d/382f6cb8-6159-4e8a-b906-0898dcb44f13/e33ab7e5-3b60-4e68-b485-04ac07b726a0.jpg" title="Tap to Download"/><br /></div>
<div><br /></div>
<div>Other strategies that didn’t work:</div><ul><li><div>Applying to jobs online but using a Talk Hiring flyer as my resume. I got no bites and was surprised how many times I got calls from recruiters about how great a fit I was for the position.</div></li><li><div>General networking groups. I got 2 free pilots out of networking groups, but most members are solopreneurs/extremely small businesses (< 5 people companies). I met some great people that are very helpful to the business to this day, but it just wasn’t a great customer acquisition channel.</div></li></ul><div><br /></div>
<div>On a whim, I decided to check out a job fair in Trenton, NJ. That was the start of our job fair strategy. Some job fairs are ideal places to find initial beta testers of an HR tech product. But I stress *some*. The best job fairs for getting face time with hiring managers are free ones (run by the government) located in suburban areas. Job fairs in NYC are overcrowded with lines snaking around the block. You want a job fair that isn’t too busy so that hiring managers are bored enough to talk to you 🙂. And, not surprisingly, free job fairs have the most employers. I’m a Jersey kid, so this <a href="http://lwd.state.nj.us/WorkForceDirectory/jobfair.jsp">job fair website</a> was my bible (got to love Mapquest directions for every event). I would walk in with a stack of resumes in a folder as to not look suspicious, and have a stack of Talk Hiring flyers below.</div>
<div><br /></div>
<div>I was able to pilot our first product, an automated phone screening tool, with a custom kitchen range hood company, a lead generation company, a home health aide business, a gym, a chess education company, a car wash, a day care, and a few restaurants. </div>
<div><br /></div>
<div>At these job fairs, I would meet people that worked at job training programs. Job training programs are always looking to enroll unemployed/underemployed individuals in their programming. Plus, I knew a lot about this industry through volunteering with a job training program in Brooklyn for a year. </div>
<div><br /></div>
<div>I heard "can we use this tool for interview practice?" a few too many times at these job fairs. I decided to run an experiment and reconfigure the tool into a mock interviewing tool. We’ve been focusing on our mock interviewing tool ever since.</div>
<div><br /></div>
<div><br /></div>
<div><br /></div>
So you want to run a study?https://harrisosserman.postach.io/post/so-you-want-to-run-a-study2020-05-14T03:03:08.063000Z2019-06-23T23:09:50ZHarris Osserman<div>We ran a study to measure if using the <a href="https://talkhiring.com">Talk Hiring</a> mock interviewing tool improves interviewing skills, and we have proven that it does! <a href="https://docs.google.com/document/d/1sqLllYxWRr9UV6WQd86SKwz5zGVY26utCNLHx2mSqaI/edit?usp=sharing">Here’s a link to our study setup and results.</a> Running this study was much harder than I thought it would be. Here are my learnings for others who might want to run their own study! </div>
<div style="text-align: center; "><img src="https://cdn-images.postach.io/d7832041-1d7e-4ec7-bb7e-099193cc8c7d/c670a028-94e7-417b-a29e-84951fe8d34e/02091840-5cb1-4f42-a65f-bf5748c02c0e.jpg" title="Tap to Download" width="435"/><br/></div>
<div><span style="font-weight: bold;">Why did we run a study?</span> Our product helps job seekers become better at interviewing. To make a claim like that, we wanted to back it up with data. First we thought, why not measure hiring outcomes? There are too many confounding variables to measure hiring outcomes from people who use or do not use our tool; plus, it would take much longer to prove or disprove. Then we thought, can we use our existing interview data to measure interviewing improvement? Users practice with over 100 different interview questions across different industries. It’s hard to compare interview quality across different questions, let alone across different industries. We decided to run a study so that we could control variables and prove the worth of our mock interviewing product. </div>
<div style="text-align: center; "><img src="https://cdn-images.postach.io/d7832041-1d7e-4ec7-bb7e-099193cc8c7d/c670a028-94e7-417b-a29e-84951fe8d34e/544a5cba-1980-438a-a4e7-1507b5980a6a.jpg" title="Tap to Download" width="380"/><br/></div>
<div><span style="font-weight: bold;">Recruiting Participants</span>: As the operator of a self-funded startup, I initially tried to run the study without paying participants. <a href="https://talkhiring.com">Talk Hiring</a> works with workforce development organizations mostly in NYC, and I tried to recruit participants through those organizations. We were able to recruit a few people through those channels, but the study would have dragged on for months if we had continued down that path. I tried running free mock interviews with people, and using those interviews as data points in the study, but that was taking up so much of my time with lots of people canceling/rescheduling. I finally decided to recruit for the study via a $10 Craigslist posting and through my existing Talk Hiring user base.</div>
<div><br/></div>
<div><span style="font-weight: bold;">A Note about Craigslist</span>: If your participants can complete the task remotely, always set the location of your Craigslist task to a major metropolitan location. We had people from all over America completing our study because they were searching for paid studies in Manhattan. </div>
<div style="text-align: center; "><img src="https://cdn-images.postach.io/d7832041-1d7e-4ec7-bb7e-099193cc8c7d/c670a028-94e7-417b-a29e-84951fe8d34e/727a628f-f87d-4870-b243-9cd8da350f2e.jpg" title="Tap to Download" width="368"/><br/></div>
<div><span style="font-weight: bold;">Money</span>: I paid every study participant $5 via PayPal. Even for that relatively small sum of money, people really needed the money. I tried to pay everyone at the end of each day, but I got pushback. A few people told me that they were planning on using my $5 for lunch that day. Others would email me in all caps right after they completed the study, asking when they would get paid. Some told me that they never received the money, and I had to screenshot my PayPal receipt to show them that they had been paid. Interacting with so many people who needed my meager $5 was disheartening.</div>
<div style="text-align: center; "><img src="https://cdn-images.postach.io/d7832041-1d7e-4ec7-bb7e-099193cc8c7d/c670a028-94e7-417b-a29e-84951fe8d34e/8388954e-0a21-4116-8717-724b82269cb1.jpg" title="Tap to Download" width="362"/><br/></div>
<div><span style="font-weight: bold;">Choosing the Right Significance Test</span>: There are lots of statistical tests of significance, and it has been 10 years since my high school stats class. I did more than my fair share of Googling to figure out how to structure our study and which significance test to use. Initially, my plan was to use a one-sided z-test to measure if a participant’s interviewing scores improved by using the tool. Once the study was underway, I realized that it was not possible to accurately score interviews on a 1-5 scale. Instead, I decided to measure simply if a participant’s 3rd try at an interview question was an improvement over their 1st try, defaulting to "no improvement" if it wasn’t clear. I ended up using a single proportion t-test to measure statistical significance.</div>
<div style="text-align: center; "><img src="https://cdn-images.postach.io/d7832041-1d7e-4ec7-bb7e-099193cc8c7d/c670a028-94e7-417b-a29e-84951fe8d34e/fd352208-2868-4be4-ba15-dd81bc9215cc.jpg" title="Tap to Download" width="418"/><br/></div>
<div><span style="font-weight: bold;">Unexpected benefits: </span>24 people participated in our study. Out of the 24 participants, 2 people sent me their resumes for employment opportunities, and 4 people told me that they would spread the word about our mock interviewing tool! </div>
<div><br/></div>
<div>A special shoutout to <a href="https://www.linkedin.com/in/joycecahoon/">Joyce Cahoon</a> and <a href="https://www.linkedin.com/in/sam-waters-644a5a28/">Sam Waters</a> for sanity checking all of my statistics in this study! </div>
<div><br/></div>
<div> </div>
Foundations of Deep Learning: Activation Functionshttps://harrisosserman.postach.io/post/activation-functions2017-07-01T18:24:35.595000Z2017-03-11T17:13:42ZHarris Osserman<div>First blog post on the x.ai blog about Activation Functions in Neural Networks. You can read it <a href="https://medium.com/@xdotai/aspects-of-deep-learning-activation-functions-fb55d72bbff3">here on Medium</a> or <a href="https://x.ai/aspects-of-deep-learning-activation-functions/">here on the x.ai blog</a></div>
<div><br/></div>
Ruthless Prioritizationhttps://harrisosserman.postach.io/post/ruthless-prioritization2017-09-29T02:40:00.256000Z2017-02-19T14:26:27ZHarris Osserman<div><font style="font-size: 18px;">Prioritization is one of the most complicated, contentious, and crucial aspects of building a startup. There’s always an order of magnitude more tasks that your team can take on. Making sure that your team is working on the highest value work items is vital.</font></div>
<div><font style="font-size: 18px;">HBR (Harvard Business Review) just released a phenomenal article on prioritization. I’ll highlight the key points here and analyze them by pulling from my startup experience.</font></div>
<div><font style="font-size: 18px;"><br/></font></div>
<div><b><font style="font-size: 18px;">The 3 Interdependent Variables in Prioritization</font></b></div>
<div style="text-align: left;"><i><font style="font-size: 18px;">"<span style="color: rgb(34, 34, 34); orphans: 2; widows: 2;">There are three interdependent variables that are essential for executing any initiative — objectives, resources, and timing. You can’t produce the desired effect of a project without precise objectives, ample resources, and a reasonable time frame. If you push or pull on one leg of this triangle, you must adjust the others. All three variables are important, but resources reign supreme"</span></font></i></div>
<div><font style="font-size: 18px;">This makes sense of course, but is very easy to forget. By speeding up a project, you’ll pull on more resources and clearly alter the importance of certain objectives. If you shift objectives, you’ll shorten the timing for some initiatives, but increase the timing for others (while adjusting your allocation of resources). If you hire or fire employees, the timing of your projects and the importance of objectives will necessarily change. To engineers, this will likely remind you of the <a href="https://en.wikipedia.org/wiki/CAP_theorem">CAP Theorem</a>. </font></div>
<div><font style="font-size: 18px;">The reason why resources “reign supreme" is because they are the leg of the three legged stool that actually gets things done. Without having dedicated resources, objectives and timing just don’t matter. </font></div>
<div><font style="font-size: 18px;">This is especially true in startups because the team is tiny. A few crucial team members leave the company, and management has to quickly shift priorities and timing of projects. Similarly, making the hiring plan more aggressive causes very obvious and immediate changes to what the team can get done in a quarter.</font></div>
<div><font style="font-size: 18px;"><br/></font></div>
<div><b><font style="font-size: 18px;">The 3 Levels of Priority: Critical, Important, and Desirable</font></b></div>
<div style="orphans: 2; widows: 2;"><font style="font-size: 18px;"><i>"<span style="color: rgb(34, 34, 34); font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); float: none;">A </span><span style="box-sizing: inherit; color: rgb(34, 34, 34); font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);">critical</span><span style="color: rgb(34, 34, 34); font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none;"> priority is an objective that must be successfully accomplished within a specified amount of time, no matter what.</span></i><font color="#222222"><i>"</i></font></font></div>
<div><span style="orphans: 2; text-align: start; text-indent: 0px; widows: 2; float: none;"><font style="font-size: 18px;" color="#222222">The “no matter what" bit is the key part. Critical projects should be given unlimited resources towards achieving their goal. The manager running the project is allowed to pull resources (people and money) at will to get the project done because the timing is fixed. These are the classic deadline heavy projects.</font></span></div>
<div><span style="color: rgb(34, 34, 34); font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none;"><i><font style="font-size: 18px;">"<span style="color: rgb(34, 34, 34); font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); float: none;">An </span><span style="box-sizing: inherit; color: rgb(34, 34, 34); font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);">important</span><span style="color: rgb(34, 34, 34); font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none;"> priority, on the other hand, is an effort that can have a significant positive impact on performance. For these initiatives, resources are fixed and the variable is either time or the objective"</span></font></i></span></div>
<div style="orphans: 2; widows: 2;"><font style="font-size: 18px;" color="#222222">Important objectives are still objectives that have to get done, but the crucial part is that there is no strict time or objective pressure.</font></div>
<div style="orphans: 2; widows: 2;"/>
<div style="orphans: 2; widows: 2;"><font style="font-size: 18px;"><i>"<span style="color: rgb(34, 34, 34); font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); float: none;">A </span><span style="box-sizing: inherit; color: rgb(34, 34, 34); font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);">desirable</span><span style="color: rgb(34, 34, 34); font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none;"> priority is an effort in which resources and time are both variables. The organization desires an outcome but cannot absolutely commit specific resources over any specifiable time period.</span></i><font color="#222222"><i>"</i></font></font></div>
<div><span style="color: rgb(34, 34, 34); font-style: normal; font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none;"><font style="font-size: 18px;">Desirable objectives would go under the “nice to have" category. These could be optimizations to the current product, infrastructure improvements, etc.</font></span></div>
<div><span style="color: rgb(34, 34, 34); font-style: normal; font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none;"><font style="font-size: 18px;"><br/></font></span></div>
<div><span style="color: rgb(34, 34, 34); font-style: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none;"><b><font style="font-size: 18px;">How Resources are Affected</font></b></span></div>
<div><font style="font-size: 18px;"><span style="color: rgb(34, 34, 34); font-style: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none;">It becomes crystal clear that in this 3 tiered priority system, managers must be very careful about assigning tasks as critical. </span><span style="color: rgb(34, 34, 34); orphans: 2; widows: 2;">Critical projects can take on unbounded resources. They can’t pull resources from important projects; those still need to get done. The only option is for critical projects to take on resources from desirable projects. </span></font></div>
<div><span style="orphans: 2; widows: 2;"><font color="#222222"><span style="font-size: 18px;">At a startup, it’s very easy to throw around the word “critical" when we actually mean “important". The word critical is very clear to employees, and it can start to become a catch-all for anything that is necessary to get done. Employees feel urgency to get things done when they’re working on critical projects. But, having too many critical projects causes confusion. Certain projects that are actually important will try to poach extra resources, limiting the pool for truly critical initiatives. </span></font></span></div>
<div><span style="orphans: 2; widows: 2;"><font style="font-size: 18px;" color="#222222">Additionally, the organization can start to feel what I call “critical burnout". If everything is critical, then nothing is. We become desensitized to what working on a critical task really means. This is a disaster. There will actually be critical things to get done, and it’ll be harder to get your team to mobilize if it feels like just another work item. </font></span></div>
<div><font style="font-size: 18px;"><br/></font></div>
<div><font style="font-size: 18px;">Thanks for reading! I consider myself a prioritization junkie, and I’d love to hear about your experiences with this subject and how it relates to your organization.</font></div>
<div><span style="color: rgb(34, 34, 34); font-style: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none;"><font style="font-size: 18px;"><br/></font></span></div>
<div><font style="font-size: 18px;">Link to Full Article: <a href="https://hbr.org/2017/02/a-better-way-to-set-strategic-priorities">https://hbr.org/2017/02/a-better-way-to-set-strategic-priorities</a></font></div>
<div><br/></div>
word2vec in JavaScripthttps://harrisosserman.postach.io/post/word2vec-in-javascript2017-10-29T15:40:44.110000Z2016-04-03T17:10:40ZHarris Osserman<div>One of my personal goals is to better understand deep learning. Word2vec, an algorithm developed by Google, is one of the simplest (but still not simple!) forms of deep learning. Deep Learning is a type of machine learning when there is <a href="http://stats.stackexchange.com/questions/63152/what-does-the-hidden-layer-in-a-neural-network-compute">at least one hidden layer</a>. Word2vec is an algorithm that develops vectors for words based purely on the assumption that similar words are often used in similar ways. By running words through this algorithm, you can find a vector for a word, and then take the cosine between 2 vectors to find <a href="http://blog.christianperone.com/2013/09/machine-learning-cosine-similarity-for-vector-space-models-part-iii/">how similar the words are…so cool</a>! There are 2 ways that word2vec can be implemented.</div>
<ul>
<li>CBOW (Continuous Bag of Words) - The algorithm is given a context (n words before and after the word), and optimizes for the middle word. This is the strategy that I used and implemented.</li>
<li>Skip Gram - The algorithm is given the middle word and optimizes for the context</li>
</ul>
<div><br/></div>
<div>The gist of the CBOW word2vec algorithm is the following:</div>
<ol>
<li>Start with an enormous corpus. An often used corpus is called text8 (<a href="http://mattmahoney.net/dc/textdata">http://mattmahoney.net/dc/textdata</a> and search for text8.zip). I copied it into corpus.txt by using <code><span style="color: rgb(0, 0, 0); font-family: Raleway, sans-serif; font-size: 14px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: left; text-indent: 0px; text-transform: none; white-space: normal; widows: 1; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; background-color: rgb(255, 255, 255);">head -1 text8 > corpus.txt</code>. This command puts everything on the first line of the text8 file into corpus.txt. text8 is a huge 1 line file, so it works out.</span></li>
<li>Assign random values between [-1, 1] for all of the entries in W, W’, and h.</li>
<li>Generate a map D and DPrime. D is a map with every word in the vocabulary as the key and with a list of each context that it appears in as the value. DPrime is a map where each key is a word and its value is a list of some contexts that <b>DO NOT</b> appear in the text for that word. When Google was developing word2vec, they found that it was important to both optimize words and contexts that SHOULD go together, as well as “teach" the network about words and contexts that should <b>NEVER</b> go together.</li>
<li>Run each context in D and DPrime through the network below <i>TRAINING_ITERATIONS</i> times.</li>
</ol>
<div><br/></div>
<div>The architecture of the one hidden layer network:</div>
<div><img src="https://cdn-images.postach.io/d7832041-1d7e-4ec7-bb7e-099193cc8c7d/097950f1-2231-48cb-9c3a-dcefa8ea3001/f297b956-90f2-4c29-9208-12be0050e819.png" style="height: auto;"/></div>
<ol start="5">
<li style="display:inline;list-style:none;">
<ul>
<li>There are V unique words in the vocabulary and the hidden layer (single dimension vector h) has length N </li>
<li>With CBOW, vector x becomes a vector with a 1 in all places where the context word exists and a 0 everywhere else. </li>
<li>In my script, I arbitrarily used an N of size 100. All other constants that I chose are at the top of my script.js file as well.</li>
</ul>
</li>
<li>In each iteration of the learning loop:
<ol>
<li>Create intermediate vector Vc:
<ol>
<li>The x vector is 1 at the indices of the context and 0 everywhere else. Therefore, when you multiply W transpose * X, the output will be the columns of W transpose that have the indices of the context. This is equivalent to the rows of W that have the indices of the context words. Vc is a vector with N rows and 1 column because that is the output of doing W transpose * X</li>
</ol>
</li>
<li>Create intermediate vector Vw:
<ol>
<li>Vw is the column of the W’ matrix (or the row of the W’ transpose matrix) that corresponds to the middle word</li>
</ol>
</li>
<li>Find the intermediate output. This is the output that you are optimizing in each iteration of the loop. The following equations come from taking the derivative of the <a href="https://en.wikipedia.org/wiki/Maximum_likelihood<a href="https://en.wikipedia.org/wiki/Maximum_likelihood">https://en.wikipedia.org/wiki/Maximum_likelihood</a>">Maximum Likelihood Estimation</a>.
<ol>
<li>If we are looking at a middle word and context that is not in the corpus:
<ol>
<li>Log(1 / (1 + e^(Vw x Vc))) </li>
</ol>
</li>
<li>If we are looking at a middle word and context that is in the corpus:
<ol>
<li>Log(1 / (1 + e^(Vw x Vc x (-1)))) </li>
</ol>
</li>
</ol>
</li>
<li>Use <a href="https://en.wikipedia.org/wiki/Gradient_descent">gradient ascent</a> to update each element in the W and W’ matrix that was affected by this iteration. The rows of W that are affected are the rows in the context, and the column of W’ that is affected is the column corresponding to the middle word. Gradient ascent is used to maximize a function, and in this case we are trying to maximize the likelihood that the middle word is the word in between the context. Gradient descent is used when trying to minimize a function. </li>
</ol>
</li>
</ol>
<div><br/></div>
<div>In every <a href="http://stackoverflow.com/questions/31155388/meaning-of-an-epoch-in-neural-networks-training">epoch</a> of the loop, I update all of the word vectors by writing to a csv file, as well as writing a <code>hello.txt</code> that finds the 10 words that are most similar to the word hello. We find word similarity by finding the 10 words with the largest cosine similarity.</div>
<div><br/></div>
<div>I hope my JavaScript is simple and easy to understand <span style="-evernote-sentoj-emoticon:true">😃</span> <a href="https://github.com/harrisosserman/word2vec">https://github.com/harrisosserman/word2vec</a></div>
<div><br/></div>
<div>The best materials that I found on the topic are these 2 YouTube videos from University of Waterloo Professor Ali Ghodsi. Thanks so much for your lectures!</div>
<div><a href="https://www.youtube.com/watch?v=TsEGsdVJjuA"/>https://www.youtube.com/watch?v=TsEGsdVJjuA</div>
<div><a href="https://www.youtube.com/watch?v=nuirUEmbaJU">https://www.youtube.com/watch?v=nuirUEmbaJU</a></div>
<div><br/></div>
<div>After writing the algorithm, I enjoyed reading this paper to tune and optimize my word vector output: <a href="https://levyomer.files.wordpress.com/2015/03/improving-distributional-similarity-tacl-2015.pdf">https://levyomer.files.wordpress.com/2015/03/improving-distributional-similarity-tacl-2015.pdf</a></div>
<div><br/></div>
Alarm Clock iOS Apphttps://harrisosserman.postach.io/post/alarm-clock-ios-app2015-07-08T03:01:25.632000Z2015-07-08T02:04:04ZHarris Osserman<div>I share my bathroom with a roommate. The worst thing that can happen in the morning is to wake up and find my roommate in the shower. To solve this problem, I built an Alarm Clock app that shares my wake up time with my roommate. That way, I can know when my roommate is planning on waking up, and I can set my wake up time accordingly. </div>
<div><br/></div>
<div>I use UILocalNotifications to do the wake up. These notifications work great because an internet connection is not required for the notification to fire and I don’t need to manage running a background job. To do authentication, I used Twitter’s Digits SDK, which works seamlessly for doing phone number based authentication. Having an email address and password are not required to create an account on my app. For all my database storage, I used Parse. I did not want to manage a server for this small application. </div>
<div><br/></div>
<div>All of the source code is here: <a href="https://github.com/harrisosserman/AlarmClock">https://github.com/harrisosserman/AlarmClock</a></div>
Building a PDF in iOShttps://harrisosserman.postach.io/post/building-a-pdf-in-ios-with-storyboards2015-07-08T02:57:49.246000Z2015-03-08T20:59:51ZHarris Osserman<div>Storyboards in Xcode are a great way to build PDFs. But it can be challenging to make PDFs with high resolution without knowing the following trick.</div>
<div><br/></div>
<div style="text-align: center"><img src="https://cdn-images.postach.io/d7832041-1d7e-4ec7-bb7e-099193cc8c7d/aafe9c47-745c-4390-bd16-82302ec6e774/f00f1d81-d99d-4ee0-93a5-f8ad4bce1c17.png" style="height:auto;" width="497"/><br/></div>
<div><b>Scaling the PDF</b></div>
<div><b><br/></b></div>
<div>By default, the text resolution for a printed PDF generated by a Storyboard will be mediocre. To make the text much crisper, I recommend the following:</div>
<div><br/></div>
<ol>
<li>Double or quadruple all of the measurements on the design spec. </li>
<li>Build the Storyboard to those specifications</li>
<li>Before rendering the PDF, shrink all of the elements by 2-4 times.</li>
</ol>
<div><br/></div>
<div>In the following code block, I am halving the measurements of the view. Halving the measurements quadruples the DpI (Dots per Inch). We are doubling the number of dots in each dimension, which results in a fourfold increase in DpI.</div>
<div><br/></div>
<div>
<pre style="box-sizing: border-box; padding: 0.5rem; font-family: Monaco, Menlo, Consolas, 'Courier New', monospace; font-size: 0.75rem; color: rgb(51, 51, 51); border-top-left-radius: 4px; border-top-right-radius: 4px; border-bottom-right-radius: 4px; border-bottom-left-radius: 4px; word-break: normal; word-wrap: break-word; white-space: pre-wrap; background-color: rgb(251, 250, 248); border: 1px solid rgba(0, 0, 0, 0.14902); font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-position: initial initial; background-repeat: initial initial;">
//First PDF Page<br style="box-sizing: border-box;"/> UIGraphicsBeginPDFPage()<br style="box-sizing: border-box;"/> CGContextSaveGState(pdfContext)<br style="box-sizing: border-box;"/> CGContextConcatCTM(pdfContext, CGAffineTransformMakeScale(0.5, 0.5))<br style="box-sizing: border-box;"/> let firstPagePdfStoryboard = UIStoryboard(name: "GameStatsPDFCoverPage", bundle: nil)<br style="box-sizing: border-box;"/> let firstPagePdfVC = firstPagePdfStoryboard.instantiateViewControllerWithIdentifier("GameStatsPDFCover")<br style="box-sizing: border-box;"/> firstPagePdfVC.view.layer.shouldRasterize = false<br style="box-sizing: border-box;"/> firstPagePdfVC.view.layer.renderInContext(pdfContext)<br style="box-sizing: border-box;"/> CGContextRestoreGState(pdfContext)
</pre></div>
<div><br/></div>
<div>At GameChanger we experienced this problem. We were building a rich PDF for basketball coaches to share with players, fans, and the press. By scaling up and then scaling down all the view elements, we were able to build a very high quality PDF. Here it is:</div>
<div><br/></div>
<div> <a href="https://cdn-files.postach.io/d7832041-1d7e-4ec7-bb7e-099193cc8c7d/aafe9c47-745c-4390-bd16-82302ec6e774/89cccfed-7eeb-4dc5-bb30-71ee4d6286c3.pdf" target="_blank">Download PDF</a></div>
MailBadger: A Postmortemhttps://harrisosserman.postach.io/post/mailbadger2017-10-29T15:52:26.933000Z2015-01-14T03:24:11ZHarris Osserman<div>
<div dir="ltr" style="border: 0px; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 15px; font-style: normal; font-weight: 300; outline: 0px; padding: 0px; vertical-align: baseline; color: rgb(55, 55, 55); font-variant: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);">
</div>
<div>At the start of junior year at Duke, I was working on a startup called MailBadger with my friend, Kirill Klimuk. Essentially, MailBadger was a two-fold enterprise software solution. It helped companies automatically follow-up with their clients, and it displayed these results in a simple, user-friendly interface organized by client or by reminder. MailBadger was going to solve the problem of requiring employees to remind themselves to follow-up with their clients. Instead of reminding themselves to remind others, employees would simply set a reminder with MailBadger for their clients, and MailBadger would send an email reminder at the specified date and time.</div>
</div>
<div dir="ltr" style="border: 0px; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 15px; font-style: normal; font-weight: 300; outline: 0px; padding: 0px; vertical-align: baseline; color: rgb(55, 55, 55); font-variant: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);">
<div><br/></div>
</div>
<div dir="ltr" style="border: 0px; font-style: normal; font-weight: 300; outline: 0px; padding: 0px; vertical-align: baseline; font-variant: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);">
<div><span style="color: rgb(55, 55, 55);"><span style="font-size: 15px;">We were interested in helping accountants that handle taxes. Many accountants that we talked to echoed the organizational challenges of being an accountant. During tax season, it is very time-intensive to make sure that all of an accountant’s clients have sent the accountant all of the legal forms that the accountant needs to do the taxes. What often happens is that a secretary will keep a huge Excel document with client names on one axis and documents required on the other axis. Then, the secretary will mark off which clients have submitted which documents. When clients do not submit documents in a timely manner, the secretary has to constantly follow up until the client has sent all of the required documents.</span></span></div>
<div><br/></div>
</div>
<div dir="ltr" style="border: 0px; font-style: normal; font-weight: 300; outline: 0px; padding: 0px; vertical-align: baseline; font-variant: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);">
<div><span style="font-size: 15px;"><span style="font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif;"><span style="color: rgb(55, 55, 55);">Kirill and I saw accounting as a huge opportunity to digitize the receipt of documents and automate the following up with clients to send required documents. However, we realized very quickly that CPAs are slow to change, are not particularly skilled with technology, and value the personal connections that they have with their clients. Many accountants are Excel power users, and do not like using products that are not tied into Excel. Additionally, because many accounting firms are run by multiple partners, it can be difficult to get all of the partners to agree on a new technology product to implement throughout the company. Finally, many clients stay with accountants because of the personal connection that the clients have with their accountants. Some accountants saw MailBadger as a product that would make communication less personal, and could translate into lost clients at an accounting firm.</span></span></span></div>
</div>
NSNotification Debthttps://harrisosserman.postach.io/post/nsnotification-debt2015-04-04T02:53:50.635000Z2015-01-07T01:50:09ZHarris Osserman<div>**Disclaimer: NSNotifications are not iOS push notifications**</div>
<div><br/></div>
<div>NSNotifications are signals that can be passed globally between classes in an iOS project. Classes can send (or post) notifications to classes that listen for the notification. For more information, check out the documentation <a href='https://developer.apple.com/library/ios/documentation/Cocoa/Reference/Foundation/Classes/NSNotification_Class/index.html'>here</a></div>
<div><br/></div>
<div><img src="https://cdn-images.postach.io/w600_a63b470b6300e7904cff1467b2bce9bc.jpg" style="height: auto;"/><br/></div>
<div><br/></div>
<div>Extensive use of notifications are bad for a number of reasons:</div>
<div><br/></div>
<div><b>Notifications lead to poor software architecture </b></div>
<div><br/></div>
<div>When building an iOS app, it’s very tempting to use NSNotifications to pass around signals. Just because it is easy doesn’t mean that you should use them often. Notifications are easy to use because the sender does not need to hold a reference to the receiver. Therefore, your app could be poorly designed (from a software architecture point of view), but held together by crisscrossing notifications.<br/></div>
<div><br/></div>
<div>
<div><b>Each notification can become gradually larger scoped than was initially desired</b></div>
</div>
<div><b><br/></b></div>
<div>It’s critical that each notification’s usage is defined. Otherwise, it is very easy for a notification to become used for more than it was designed for. For example, let’s say there is a ‘userChanged’ notification that would get fired every time that the user’s information was changed. A name like this is often too vague because a lot of different things can change for a user. Therefore, you will probably end up reloading/recalculating more parts of your app than are necessary, hurting the app’s performance. A better name would be something like ‘userChangedContactInfo.'</div>
<div><b><br/></b></div>
<div><b>It’s hard to debug an app with lots of notifications</b></div>
<div><b><br/></b></div>
<div>When a notification is fired, more than 1 method can (and often is) executed. Multiple code paths need to be investigated, which is annoying and time-consuming.</div>
<div><br/></div>
<div><br/></div>
<div><br/></div>
<div> </div>
Pathhttps://harrisosserman.postach.io/post/a-product-i-love-path2015-04-04T03:02:41.435000Z2015-01-04T21:42:55ZHarris Osserman<div><div dir="ltr"><span style="background-color: transparent; vertical-align: baseline;"><span style="white-space: pre-wrap;">Path</span> <span style="font-weight: normal; color: rgb(0, 0, 0); font-style: normal; font-variant: normal; white-space: pre-wrap; text-decoration: none; font-size: 15px;"><span style="font-family: Arial;">is a mobile only social network for your closest friends. It provides the basics of a social network (messaging, news feed, picture/video posting, location check-ins, status updates, and “friending" of people). But, because it is more intimate than a larger social network (ex. Facebook), it has become a much richer and detailed way to document my life. </span></span></span></div></div>
<div dir="ltr" style="text-align: center"><img src="https://cdn-images.postach.io/w600_78fff20ede5b987fc290f44e8d4c9c99.jpg" style="height:auto;" width="182"/></div>
<div dir="ltr" style="text-align: center;"><b><span style="font-size: 15px;"><span style="font-family: Arial;"><br/></span></span></b></div>
<div dir="ltr" style="text-align: center;"><span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:bold;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Things I Love</span></div>
<div dir="ltr"><span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Path is known for its design. Path’s best design elements are the clock, which changes as the user scrolls, and the “+" button that allows the user to add a new moment. Moving clock hands as the user scrolls is a truly innovative UI element, and is something that I have not seen on any application. Pressing the “+" button beautifully releases five different buttons that the user can select.</span></div>
<div dir="ltr"><span style="font-size: 15px;"><span style="font-family: Arial;"><br/></span></span></div>
<div dir="ltr"><span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">My favorite features in Path are emoticons, and book/movie/music/tv show sharing. Path allows emoticon posting on moments. Unlike Facebook where users are only able to ‘like’ a post, posts (or moments) on Path can be smiled at, winked at, frowned at, surprised at, or loved. This allows for rich and quick responses to a friend’s moment, while adding a unique feature to social networking. </span></div>
<div dir="ltr"><span style="font-size: 15px;"><span style="font-family: Arial;"><br/></span></span></div>
<div dir="ltr"><span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">I really enjoy book/movie/music/tv show sharing. My friends have shared lots of great media through Path that I did not know about, and I have discovered new media this way. Path makes it easy to share what music is currently playing on a user’s smartphone, and has autocompletion for most music, movies, books, and tv shows. When friends share songs they are listening to, other friends are able to listen to a preview of the song in the app, and when friends share movies or books, Path displays a short description of the movie or book in the app. The intimacy of Path promotes more sharing than on Facebook, and has allowed me to better stay in touch with my closest friends.</span></div>
<div dir="ltr"><span style="font-size: 15px;"><span style="font-family: Arial;"><br/></span></span></div>
<div dir="ltr" style="text-align: center"><b><span style="font-size: 15px;"><span style="font-family: Arial;">Things I Don't</span></span></b></div>
<div dir="ltr"><span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">One feature I never use is the ability to share sleep information. I have shared sleep information with friends, and every time friends either comment on me sleeping too much or me sleeping too little. The predictability of the responses to my hours of sleep gets old, and I no longer record my sleep data. I have approximately 30 friends on Path, and none of them share sleep data either. Sharing how many hours you sleep every day seems to be oversharing, and is an unnecessary feature of the application, in my opinion.</span></div>
<div dir="ltr"><br/></div>
<div>
<p dir="ltr" style="text-align: center;"><span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:bold;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">A Way to Monetize Path</span></p>
</div>
<div dir="ltr"><span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Path is currently trying to monetize by selling virtual goods, either as a monthly/yearly subscription or on a per item basis. This is a great start, but I am skeptical that this tactic will make Path enough money. I believe that sponsored stickers would improve the user experience while making Path and advertisers happy. </span></div>
<div dir="ltr"><span style="font-size: 15px;"><span style="font-family: Arial;"><br/></span></span></div>
<div dir="ltr"><span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">There are limitless options for sticker packs, and these would make the app much more fun while bringing in more money for Path. Sponsored sticker packs would be free for users, and would add more sticker variety for users to post. I know that Path is trying to build a social network without ads, but I think that these ads would not be intrusive at all. </span></div>
My favorite books of 2014https://harrisosserman.postach.io/post/my-favorite-books-of-20142015-04-04T02:54:10.325000Z2015-01-04T20:33:50ZHarris Osserman<div>I made a lot of time to read in 2014. Here were my favorite books:</div>
<div><br/></div>
<ul>
<li><a href='http://www.amazon.com/Hatching-Twitter-Nick-Bilton-ebook/dp/B00CDUVSQ0/ref=sr_1_1?ie=UTF8&qid=1420426390&sr=8-1&keywords=hatching+twitter'>Hatching Twitter</a> — Incredible read about the dysfunctional founding team of Twitter. I couldn’t put it down…definitely a page turner</li>
<li><a href='http://www.amazon.com/Rosie-Project-Don-Tillman-ebook/dp/B00BSBR9N6/ref=sr_1_1?ie=UTF8&qid=1420426439&sr=8-1&keywords=rosie+project'>The Rosie Project</a> — Fictional book about a nerdy guy who tries to find love by giving out surveys to bachelorettes. He tries to apply the scientific method to love and fails, but in the process, finds love unconventionally.</li>
<li><a href='http://www.amazon.com/Ending-Aging-Rejuvenation-Breakthroughs-Lifetime-ebook/dp/B001ANSSKA/ref=sr_1_1?ie=UTF8&qid=1420426462&sr=8-1&keywords=ending+aging'>Ending Aging: Breakthroughs That Could Reverse Human Aging In Our Lifetime</a> — Aubrey de Grey argues that our view towards aging is wrong. We strive for cures to cancer, diabetes, stroke, etc, but are fine watching our bodies age. Aging is just an accumulation of damage in the body. Aubrey gives lots of examples of how we could stop aging, as well as the status of those medical breakthroughs.</li>
<li><a href='http://www.amazon.com/10%25-Happier-Reduced-Self-Help-Actually-ebook/dp/B00FJ376CS/ref=sr_1_1?ie=UTF8&qid=1420426483&sr=8-1&keywords=10%26+happier'>10% Happier</a> — Dan Harris, a TV anchor on Good Morning America, had a panic attack on air. He decided to get his anxiety under control with meditation, and claims that meditating every day makes you 10% happier…I agree</li>
<li><a href='http://www.amazon.com/Goldfinch-Donna-Tartt-ebook/dp/B00BAXFECK/ref=sr_1_1?ie=UTF8&qid=1420426508&sr=8-1&keywords=the+goldfinch'>The Goldfinch</a> — An orphaned child steals an artistic masterpiece. He travels around America, from NYC to Las Vegas, having life-defining experiences while always holding on to his beloved Goldfinch painting.</li>
<li><a href='http://www.amazon.com/Throwback-Big-League-Catcher-Really-Played-ebook/dp/B00GEU7422/ref=sr_1_1?s=books&ie=UTF8&qid=1420426530&sr=1-1&keywords=throwback+jason+kendall'>Throwback</a> — Jason Kendall, an MLB catcher, explains baseball from an insider’s angle. I enjoyed reading about what the catcher/pitcher signs and signals actually mean.</li>
<li><a href='http://www.amazon.com/Waste-Uncovering-Global-Food-Scandal/dp/0393068366/ref=sr_1_2?s=books&ie=UTF8&qid=1420426549&sr=1-2&keywords=waste'>Waste: Uncovering the Global Food Scandal</a> — Everyone knows that there is food waste, but I never understood just how much food is wasted in all steps of the food production process. Farmers waste crops when they don’t look the way consumers expect them to. Food production plants throw out heaps of excess production when supermarkets won’t take their supply. Supermarkets throw out food past the sell by date, even though the food is safe and edible.</li>
<li><a href='http://www.amazon.com/Schulz-Peanuts-Biography-David-Michaelis/dp/0060937998/ref=sr_1_1?s=books&ie=UTF8&qid=1420426574&sr=1-1&keywords=schulz+and+peanuts'>Schulz and Peanuts</a> — Howard Schulz, the inventor and creator of Charlie Brown, struggled mightily in his early life. After each failure, he kept drawing, eventually resulting in the most successful cartoon series ever.</li>
</ul>
My first business ever: KoalaCabhttps://harrisosserman.postach.io/post/my-first-business-ever-koalacab2017-10-29T15:43:54.680000Z2015-01-04T19:48:32ZHarris Osserman<div>During the summer after my freshman year at Duke, my friend and I started our first business. I built my first website. It was a taxi sharing and taxi booking website called KoalaCab (I know Uber now does this, but it was pre-Uber). We ended up booking 450 cabs during our first year of operation.</div>
<div><br/></div>
<div style="text-align: center"><img src="https://cdn-images.postach.io/w600_42c5f50b41c50c1b8097e1707a5a7062.png" width="142" style="height:auto;"/><br/></div>
<div><br/></div>
<div>Colleges are great places for taxi sharing because:</div>
<ol>
<li>Students are price sensitive and want to spend as little as possible</li>
<li>Students are generally coming from/going to the same places (especially the airport before and after school breaks)</li>
<li>Students want to meet new people</li>
</ol>
<div><br/></div>
<div><br/></div>
<div style="text-align: center">This was our slogan and may have been the best part of the business:<br/></div>
<div style="text-align: center"><b style="font-weight:normal;"><img src="https://cdn-images.postach.io/w600_700ad920bb20db23030eb0aaf5d7018c.png" width="671px;" height="58px;" style="height: auto;"/></b><br/></div>
<div><br/></div>
<div><b>Building the Website</b></div>
<div><b><br/></b></div>
<div>At the end of freshman year, I had written very little code before, and had no idea how websites worked. I saw a tutorial on Joomla called Website Essential Training, and it taught me the basics of how to build a website with PHP and MySQL. Here is version 1 of the website:</div>
<div><br/></div>
<div style="text-align: center"><img src="https://cdn-images.postach.io/w600_2bd09c07734558dc6a486226a275029b.png" style="height:auto;" width="385"/><br/></div>
<div style="text-align: center"><br/></div>
<div>I ended up using CodeIgniter, a PHP MVC framework, which helped with security (specifically SQL attacks) and code cleanliness. I used JQuery Mobile for the KoalaCab mobile website. In hindsight, building an app instead of a mobile website would have been the right decision, but I didn’t have the time to learn iOS/Android development. </div>
<div><br/></div>
<div style="text-align: center">Final version before I killed off KoalaCab:</div>
<div style="text-align: center"><br/></div>
<div style="text-align: center"><b style="font-weight:normal;"><img src="https://cdn-images.postach.io/w600_2e84224f8bef644761c0861ca93387ab.png" width="435" height="363px;" style="height:auto;"/></b></div>
<div><br/></div>
<div><b>Building the Business</b></div>
<div><b><br/></b></div>
<div>For KoalaCab to work, we would need to partner with a taxi company in the Duke area. Riders would book rides through our system, the taxi company would drive the passenger, and we would take a cut. We met with all the taxi companies in the Duke area to see what they would offer us. All the taxi companies were willing to partner, some on better terms than others. One company was willing to give my friend and me free rides for life (in hindsight, maybe we should have done that deal). We vetted taxi companies on their tech capabilities, their management style, promptness, and reputation among Duke students. We ultimately chose Charlene’s Safe Ride because they allowed students to pay with their FLEX school dollars.</div>
<div><br/></div>
<div><b>Learnings</b></div>
<div><b><br/></b></div>
<div>College students are very last minute, which is terrible for a web-based taxi sharing service. It’s very hard to convince students to book a cab in advance. The later the cab is requested, the harder it is for people to find out about the ride and join in. When users came to our site, they were forced to first search for a cab with a similar itinerary before booking a cab. If a cab was already doing a similar route, the user could join in. If not, the user could create a new ride. If everyone is searching/booking last minute, it is harder for people to truly cab share, and KoalaCab becomes more of a cab booking website for solo rides. </div>
<div><br/></div>
<div>And please don’t be in “stealth mode" while building your company. Nobody wants to steal your idea. Tell people about it, and have potential customers test it constantly. </div>
<div> </div>
<div style="text-align: center"><br/></div>
<div style="text-align: center"><br/></div>
Lean Startup Iterationhttps://harrisosserman.postach.io/post/when-lean-startup-iteration-doesnt-work2017-09-01T03:07:18.041000Z2015-01-04T19:26:04ZHarris Osserman<div>
<div style="border: 0px; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 15px; font-style: normal; font-weight: 300; outline: 0px; padding: 0px; vertical-align: baseline; color: rgb(55, 55, 55); font-variant: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);">
</div>
<div>During my sophomore and junior years at Duke, I worked on startups. It was a time of iteration, of building and then asking customers what they want. The problem with iteration is that it often leads to dead ends.</div>
<div><br/></div>
</div>
<div style="border: 0px; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 15px; font-style: normal; font-weight: 300; outline: 0px; padding: 0px; vertical-align: baseline; color: rgb(55, 55, 55); font-variant: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);">
<div>Yes, iteration can steer a startup into a small market that it can hopefully conquer. But what I have realized is that iteration can also steer a startup into a miniscule market, or a decent market that has miniscule adjacent markets. Often, these adjacent markets are too small to expand into, which either kills the startup or causes another drastic pivot.</div>
<div><br/></div>
</div>
<div>
<p style="border: 0px; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 15px; font-style: normal; font-weight: 300; outline: 0px; padding: 0px; vertical-align: baseline; color: rgb(55, 55, 55); font-variant: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);">With little previous knowledge of the markets that I was operating in, I simply followed where the customer took me, learning tons about my market as I went. I followed customers into accounting, procurement, tech recruiting, political campaigns, local governments, and state governments.</p>
<p style="border: 0px; font-family: 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 15px; font-style: normal; font-weight: 300; outline: 0px; padding: 0px; vertical-align: baseline; color: rgb(55, 55, 55); font-variant: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255);">After chasing after all of these customers, I have decided to take a step back and learn more about markets before doing my next startup.</p>
</div>
<div style="text-align: center"><img src="https://cdn-images.postach.io/d7832041-1d7e-4ec7-bb7e-099193cc8c7d/1524ac07-ff26-4304-83ac-890db6404d74/7ff1bfe9-2dc1-47e5-a098-2c3d75547dd6.png" style="height: auto;"/></div>