➽ Why I’m Writing This Analysis
➽ A Little Background
➤1: Aligning with a Client’s Needs
➤2: The Product Must Be “Lean” and “Fit”
➤3: Systemize Your Learning Curve
➤4: “Track, Measure and Learn”
➤5: “Compare Well, Predict Better”
➤6: Choose Demographics Carefully
➤7: Personalize the User’s First Experience
➤8: “Don’t Let Emotions Drown You, Let Data Speak”
➤9: Only In-Depth Feedback Changes the Game
➤10: “Culture Matters”
➽Why I’m Writing This Analysis
Recently I moved from Los Angeles to Silicon Valley, and had the opportunity to meet some new friends. Most of them work in the tech industry, and some of them work for interesting and cool tech startups. Hanging out with some of them one day, I asked them:
“If you were the CEO of your current startup, what would you do differently to improve it?”
I asked them this because this is a key business question that I often ask myself. If I were the boss:
1) What would I do differently?
2) How could I improve business results and make a difference?
➽A little background
Due to the confidential information some of my analysis contains, I identify the company I worked for as “Company A”. This avoids legal complications.
When I first joined “Company A” in 2014, I was told I was only their second employee. They hired me to build a team for the launch of their first product. In my interview, the CEO told me that “we are a startup, so we have to have a team ready and capable of doing everything from launching the product, to ensuring good operation, implementing marketing, and maintaining channel distribution. The bad news is that this will be a very tough job; the good news is you will likely learn a lot from it. None of us have done any of this before, so we have to figure this out as a team.” Driving back after the interview, I sincerely wondered whether I was capable of taking on this challenge.
I decided to take the job. I didn’t have experience in the field, but thorough research and careful hard work saw me slowly become more successful. After only one year and when I was still learning the job, our first employee left the team. This loss was quite overwhelming and made things even more stressful, but both the CEO and I knew we had to stay focused and achieve our ultimate goal.
➤1: Aligning with a Client’s Needs
Company A provides educational consulting services to clients. They target clients whose goal is to attend a top university, but who are not qualified to do so because of their lack of TOEFL and SAT scores. The company designed a service to enhance the learning experiences of their customers. Its “Service Package” includes an “Academic Pathway” design, tutoring services, and pastoral care and financial advice to help clients adapt to life in the U.S.
The company aimed to design a package that catered to the client’s needs; unfortunately, the service proved unsuccessful. None of the first-year users renewed their contracts.
➤2: The Product Must Be “Lean” and “Fit”
Company A’s primary business objective is retention. Customers would ideally stay with the company for at least two years to fully benefit from the services package. If the company fails to follow up early enough to figure out the individual “metrics for retention” in both qualitative and quantitative ways, the company will undoubtedly lose clients’ business.
Here is the diagram of our services package. It is broken into two parts: Core Value and Additional Value.
The “Core Value” of the package is the “academic support” service. This is always constantly in demand (around 96%-100%). Our “Additional Value” is the “Life & Living” and “Activity & Seminar” services, for which demand varies on an individual basis.
From data, it appears that some clients love the “Additional Value” part (they use these services a lot), while some don’t really care for it (some clients have never used these services at all). These variables signal that the company should refine its offering by giving clients the ability to customize the services as appropriate for them.
The illustration below gives a visualization of how the service package should be composed. X represents the percentage of our additional services package (30% is the maximum percentage). These services include helping clients find housing, arranging field trips, and holding academic seminars and lectures.
Y represents how much they love the “Additional Value Part”. If we were to cut that part out, would they care? Some do, some don’t.
I believe what should have been done is to customize the clients’ own combination for the Additional Value Part and then they can pick whatever services they would like to use. In this way, they can take a full usage of our service and the clients can fully take advantage of it.
➤3: Systemize Business Learning Curves
In order to help clients achieve their academic goals, we first needed to know whether our tutoring services were improving their academic performance or not. It turned out that our clients always asked for help at what we would term the “late stage” – usually when they realized that they were going to fail the class, or were left so far behind that they had no confidence in taking the class any further.
So why don’t they come for help earlier? Firstly it’s important to recognize that our clients are still adapting themselves to a new environment, and often they don’t know what academic life look like here in the U.S. Secondly, some clients mistakenly think they can still apply their old learning behaviors and skills in the American education evaluation system. Thirdly, they often underestimate their class level and overestimate their learning capabilities.
When I worked in “Company A”, we took initiative to help them based on what we termed “academic checkpoints”. We might have improved this system by fitting to their individual learning curve as well. Below are the things we did to help at the outset, and a list of things that we should have done to improve the quality of service:
The things we did at the outset of the service:
1) Collect class requirements and professorial requirements for clients’ upcoming classes, and rank the difficulty of these classes from easy to hard accordingly.
2) For those who didn’t require our services immediately, we would go through their notes to check they were well prepared for midterms or finals. These points were the established “academic checkpoints” used by our counselor.
3) For the students who did require our services at that point, we would meet with the students weekly to ensure our tutoring service was helping. To evaluate tutors’ work and to help both the tutor and the client visualize accumulating improvements, we would have clients fill in our “improvement worksheet”.
Improvements we could have made:
1) Build a “skills assessment package”, wherein clients would be able to access and compare their specific learning advantages and disadvantages with the skill set recommended for a given class. This would make clients aware of what they might potentially face in each class, and would be the best way to prepare them for their chosen course before their academic term even begins. Moreover, we could have given them a better idea of possible study hours required for each class so they could better plan their time.
2) Meet with clients weekly to receive updates on their academic status. We might have considered asking not only qualitative questions (what are you finding challenging? how are you doing in X class?) but also have conducted quantitative surveys (what percentage of your homework are you finding difficult? are you on track for your academic target?). This would have given them a better overview of their academic career and allow them to better understand at which point they need our services.
3) For those who need our services immediately, we should have recorded the company’s own progress indicators immediately by collecting processing cards, score records, evaluation sheets, improvement breakthroughs, and quality surveys. This would have systemized the business’ learning curve, and could give more accurate comparisons across both our clients and the company’s learning processes.
➤4: “Track, Measure and Learn”
After talking to our clients for around one year, we found that the grades after the first semester with the company shape what they think of our program enormously. The end of the first semester offered the first official point at which clients would “grade” our services. This was the time we chose to prepare and deliver the client’s “strength map”.
What we should have done is to study their “strength map”, so we can evaluate their skills set early enough to set their baseline for prospective improvements so our tutors will be better assisting them using certain methods and create a better personalized learning experience with us.
The creation of the “strength map” begins with a vital question: how do professors grade their students? This differs depending on college or school in question, but the criteria can be roughly divided like so:
- Written English
- Essays/paper/review writing
- Reading comprehension & speed reading skills
- Independent learning
- Presentation skills
- Exam techniques
- Conversational skills
- Teamwork skills
- Class participation
Once we have the “strength map”, we can create a personalized version. In retrospect, we should also have created a tiered system of four levels to evaluate a student’s skill-set:
If they achieve 90+ out of 100, they will be ranked as a “Best Performer”, at Level 4 (the Highest)
If they achieve 80+ out of 100, they will be ranked as a “Great Performer”, at Level 3
If they achieve 70+ out of 100, they will be ranked as a “Good Performer”, at Level 2
If they achieve 70- out of 100, they will be ranked as a “Need Improvement”, at Level 1
For example, let us say Tony took Psychology, and achieved a B+ at the end of his first semester. This is what his strength map should look like after his midterm:
So Tony’s aggregated score on his strength map is:
Total: 80.55 (80.55%)
If 80.55 (out of 100) equals a B+ on the professor’s evaluation system, we can more readily see which skills Tony should work on to achieve an A in this subject, and even by how many points Tony has to improve.
There are two indicators that should be prioritized:
- Highest skills percentage
- Highest & lowest skills set
For this course, writing (30%) and learning skills (30%) make up the largest proportion of the required skills to do well at the class (a total of 60%). Tony’s writing is on Level 3, and his learning skills are on Level 2. While neither is bad, the company should focus on improving Tony’s “learning skills”. Admittedly even these “skills” are broad and can be subdivided into many individual parts, but the overview is helpful to provide an overall guide to both the tutor and the student. Here, for example, the tutor would know that to improve Tony’s learning skills, they should check his syllabus outline, create a “notes overview”, evaluate a professor’s teaching focus, improve Tony’s problem-solving skills, and encourage Tony to write responses to a case study.
Also of note is that Tony’s lowest level here is Reading (Level 1); this skill informs and interacts directly with the other skills important in psychology (writing, speaking), which signals that tutors should pay extra attention to improving that element.
Tony’s his highest levels here are in Speaking (Level 4) and in Personality Factors (Level 4). Company employees should advise Tony on which upcoming courses will benefit Tony most according to this evaluation of his skills.
➤5: Compare well, predict better
In order to improve a client’s strength map, we need to constantly re-evaluate and level them up via “check points”. We should, in other words, create a long term evaluation system with an achievement and reward system to keep the students motivated.
We failed to have implemented any reward system before. After reading some articles on rewarding, I have begun to realize that building a small system to keep students motivated will make improvement their habit. Below is a simple sample, a “skills improvement tracking” form for Tony’s Psychology class in the fall semester of 2016.
To increase Psychology writing skills in Prof. Scott’s class by one point, Tony needs 2.22 tutoring hours. The precise rate of improvement will vary according to circumstances and student, but this at least gives both the company and Tony a basic picture of how much they might improve: every tutoring hour Tony receives, he improves his writing skills by 0.45 points on our scale.
These ratios (2.22 hour/one point and 0.45 point/one hour) offer key indicators for Tony’s future learning process, and will provide the company with a means of studying the student’s learning behaviors, too. Based on this evaluative process, we can set rates of achievement and reward accordingly.
➤ 6: Choose Demographics Carefully
Our mission is to help clients who want to be accepted by their dream U.S. schools, but who don’t have SAT and TOEFL scores (or are too late to take these tests). To study in the U.S., all universities require international applicants to submit their TOEFL and SAT scores. For students who are too late to take these tests and don’t want to wait for the next year, the company plans to get them started as a freshman in U.S. 2-year colleges and then transfer them to their dream universities. This works because colleges here can usually waive SAT test requirements and require lower TOEFL scores for international students.
In order to transfer successfully to their dream schools, they need to keep a good GPA during their two years in college. Meeting these standards requires not only decent English to have them keep up with native students in the classroom, but also a level of academic excellence to ensure they have great academic records. So who will be our clients? Who would be interested in that pathway?
When we were figuring out our targeting group 3 years ago, we started with this targeting chart:
– Students who have SAT and TOEFL scores will apply to schools directly based on their test scores. They are doing fine within the U.S. system and don’t need our help.
– Students who have SAT but not TOEFL scores, which is rare. Taking the SAT requires very good English.
– Students who have TOEFL but not SAT scores would definitely be in our targeting group. This is far more common than vice versa.
– Students who have neither SAT or TOEFL scores. They are our main target.
Our assumptions about our demographics were partly correct and partly incorrect.
We were correct in assuming that none of our clients would have SAT scores, but we were wrong to assume that our main targeting group would be group 4 (No TOEFL, NO SAT). In reality only 30% of our clients had neither test scores, while 70% already had a TOEFL result:
As we can see from the graph above, 70% of our clients’ TOEFL scores were around the 45-50 mark. Colleges usually require a TOEFL score of 45 as their minimum language proficiency requirement; since this is the minimum requirement, however, these students will still face linguistic challenges when taking college classes. With the help of our tutoring services, their English would ideally be improved by 10-15 points within a year.
The English level of the “No TOEFL” group (which makes up 30% of our client base) can vary considerably among members, since they have yet to take their test. The TOEFL range can vary from 10 to 30, which can offer potential issues to the company. These students need to work extremely hard to catch up with the 45-50 group, and catching up can be a very tough, sometimes painful process. I remember one experience helping our client trying to catch up from a 20 score to the college level group. Both the client and I were extremely exhausted after a year of trying. Our failure to take these problems into account properly turned out to be our biggest misconception for the NO TOEFL targeting group, but what did we do wrong?
- We had an assumption that turned out to be wrong. We thought that even if a student’s English level was under 45, they would be able to catch up in this English-speaking environment because they would speak, read, and write English every day. We thought it would accordingly not be difficult for them to catch up, but it turned out that we underestimated their “processing time”. Students would fall into that group for a reason; their low scores indicated that they had many fundamental language issues that had to be fixed, while the short time span in which they were supposed to improve meant that those fundamental issues could not be properly addressed.
- We confused ourselves by combining two different groups together. A low TOEFL score means their TOEFL score is under 45 (they are actually in the lagging group), while no TOEFL means that their TOEFL score is as yet unidentified (their actual TOEFL can range anywhere from 10 to 85). Just because a student didn’t have a TOEFL score didn’t necessarily mean that their English was bad. What we needed to do was re-evaluate the actual English level of the NO TOEFL group members more quickly.
What we should have done:
1. We should have implemented our own test for those without TOEFL scores. The test would evaluate their proficiency in the language skills they need for college classes, and would establish whether their proficiency is equivalent to TOEFL 45+ and thus adequate for them to start their college courses.
2. We could have set one-on-one conversations with the students with a sub-45 score, and layout an “English training map” to show them the required progression to the ideal English level for attending college classes. We could clearly establish their level with them, and show them precisely how we would improve their English to college level.
➤7: Personalize the User’s First Experience
Since we targeted international students, most were not yet in the U.S. Since Google is banned in our local market, we used baidu.com and Baidu analytics to implement our online campaign. We ran our online marketing campaign for about 6 months; below are some changes I would have made:
How did we pick our keywords on Baidu?
– We based our keywords on top search keywords in the education industry, and their relevance to our business. In the education consulting industry, however, the viewer group can either be students or students’ parents. This made selecting keywords difficult. Moreover, because of the varying targeting groups, expectations of our landing page also theoretically vary. Parents may actively dislike the things students want from a webpage, and vice versa. Student age range is also important. Are our students in high school looking to study abroad, or are they already enrolled in college in their home country but looking for a change?
We needed to find out who they were! We should have separated these groups’ user experiences by posting a 3-second survey on our landing page. It doesn’t have to be something fancy, and could be something as simple as this:
Once the survey is filled in the company would have a far better idea of the demographics of its website. A targeted one minute video could emphasize the most important elements of the company’s service for each group.
After having finished the video, each group would visit an auto-loading website page based on their earlier identity choice. This would separate student and parent viewing experiences so that the company can focus on their individual priorities.
➤8: “Don’t Let Emotions Drown You, Let Data Speak”
Because cell phones have limited spots for advertising listings due to the size of the screen, phone ads are more expensive than computer ads. It costs a lot to place a company’s keywords on the first page of a cell phone search engine.For the same keyword, the phone spots cost almost 3 times the same spot on a computer page (on average). This is obviously a huge expense for us, particularly because at that time we allocated 70% of the marketing budget to computer ads and only designated 30% for phone ads. But what if the customer conversion rate on cell phone ads is much higher than computer ads? What if more than half of our future clients are coming from phone ads?
For the same keyword, the phone spots cost almost 3 times the same spot on a computer page (on average). This is obviously a huge expense for us, particularly because at that time we allocated 70% of the marketing budget to computer ads and only designated 30% for phone ads. But what if the customer conversion rate on cell phone ads is much higher than computer ads? What if more than half of our future clients are coming from phone ads?
We didn’t have an appropriate budget to test phone ads to truly explore phone-user demographics and establish the phone advertising customer-acquisition cost. The higher cost of the phone ads was simply prohibitively expensive for a small company like ours, and testing was therefore impossible. While it might have been useful to have data like this, it was simply unrealistic to hope that we could do so with our limited resources. Computer ads, therefore, were our priority.
Some other thoughts on ads testing:
- Our average CPC (cost per click) was way below our break-even CPC. Even if we use the ideal max CPC (which is around 70% of the break-even CPC), we were still way lower. This does at least mean we haven’t overspent our online advertising budget. It also, however, begs the critical question– what if tested more, increasing our average CPC by different percentages (25%, 50%, 70%), and pushing different popular keywords to the 1st rank? Would the conversion rate change in any significant way?
- The cost per click to profits ratio of our social media channel is around 10%, while the company partnership channel is around 18%. As we can see, the cost of a company partnership is 1.8 times that of running the social media channel. The question is thus is a company partnership worth it? What if the company prioritizes, or just works over, social media? If social media is the primary marketing focus, should the pages be run more aggressively establish a clear optimization point?
➤9: Only In-Depth Feedback Changes the Game
The company did decide to implement a partnership deal with a local company to promote our services and find clients for us. This didn’t work well.
The partnership companies from which we selected were consultancy firms located in local markets. We found some good potential candidate businesses that had different branches in different cities, and didn’t have any overlapping services with us. We thought that these companies would be perfect for helping our expansion in locales less familiar to us. Since these companies aren’t necessarily familiar with the details of our services, however, their sales team doesn’t necessarily know how to properly sell them to the local market. Moreover, these partners are unwilling to spend too much time training their own employees for other companies’ services.
Despite the fact we had salespeople working in our local office, then, we didn’t have a marketing team there, and therefore sales lacked good guidance and direction. A better move might have been to shift the entire team back to its originating market. Here’s why:
- It would be much easier to work in the same time-zone as students and potential clients. Calls can be conducted quickly and efficiently, and office time is more easily regulated.
- Clients can visit us in person. This instills consumer trust and gives the company a recognizable face.
- Sales work can be evaluated quickly and efficiently.
- The sales team can be trained more easily, and inter-departmental communication is more efficient.
- Transparency can be facilitated across departments.
- The company can receive feedback and ideas from our local channel partners to find out the best “win-win partnership model”. The team can more easily evaluate the impacts, costs, targeting groups, chance of success and time-management details of partnerships (see chart below).
- We will have opportunity to attend local events and discern the most important ones for our target demographic. The team can more easily evaluate the impacts, costs, targeting groups, chance of success and time-management details of local events, too (see chart below).
8. The team will have access to real-time testing results on the social media channel. I suggest re-evaluating our social media presence across five variables, as below:
➤10: Culture Matters
Company culture shapes its leadership style. Companies can opt for hierarchies, or for more empowering structures. A passive leadership style often generates passive, average results. An empowering approach requires staff to be more motivated, and take on active, contributive, leadership roles. It is often this approach that encourages people to work towards better solutions to company problems.
When hiring people, it is important to consider whether they could be a contributor or leader, and not merely an executor, someone who follows orders. Contributors and leaders offer exceptional talents, insights, and skillsets to improve the business. It is the presence or absence of these exceptional talents that can make or break a company.
My role at Company A required me to recruit individuals. I knew that finding a crucial contributor was an essential part for our business growth, but attracting insightful, brilliant people requires insightful, brilliant business practices. I don’t feel that Company A offered this.