I made my students 49% smarter and I can prove it

“Well looky there, you learned something!  You’re 49% smarter than you were 5 minutes ago!”  This aha! moment  occurred while teaching over 400 business students this fall.  Using Tophat in my business research instruction sessions, I was able to assess that my students did in fact learn something through my teaching.

The Challenge

Each semester I have the awesome opportunity to teach two research sessions to  over 400 sophomore business students.  The 400 students are divided into 3-4 sections, which I teach in the same day (it can be exhausting).  The first session is generally a typical 30-45 minute database demonstration, as they need to do a basic industry analysis for their first project.  For the project, all students are researching the same industry, so the tools they will need are pretty consistent and straight forward (Type an industry keyword in a search box, get some useful stuff).

The second project is a bit more challenging to teach to, as each 5-member student team can choose their own business to create.  The resources they need to successfully complete consumer demographic, local market, and competitor analyses are quite a bit more challenging to use than the sources for the first project.  The resources they need for the second project require significantly more creativity to use, as well as more brain power to interpret the data.  In the past when I have done a basic database demo of these resources for the second project, students were paying more attention to how to use the interface than they were in understanding how they might apply the data.  This was clearly demonstrated in the 20+ consultations that I held with student teams, as almost every team had questions about how to interpret the data.    The consultations were very repetitive, with each student team having the same questions.  This was not an efficient use of my time, and was certainly slowing student learning.  There had to be another way to teach them to use the data first, and the interface second.

The Setup

In order for the students to do thorough research for their projects, they really needed to deep dive into Simmons Oneview, SimplyAnalytics, and Bizminer.  I outlined my class sessions so that the students would first look at the data available from a single database and answer questions about the data.  This would be immediately followed by a demonstration of how to navigate and find the data in the specific database.

For the sessions to be relevant to their assignment,  I needed to make up a mock business concept that could adequately demonstrate how to interpret the data from the business databases.  I chose to investigate opening a store that would serve two of the three sports of mountain biking /road cycling, running, or golf.  I created a  data handout for class (pdf) with screenshots of demographic data and local market data that would be useful in researching my business concept.  I then drafted questions that would lead students to interpret the data to make decisions about which two sports my store should cater to, as well as the location of my store.  These questions were based on the types of questions they should be asking about their own business concept ideas.

I decided that a video tutorial of each database would be more efficient and consistent at demonstrating how to find the data in each individual database.  Simply pushing “play” and watching a video would help me to stay on track with my usage of class time by avoiding database/internet slowdowns and my own tangential ramblings, as might be the case during a live demonstration.  I created three videos, one for each of the essential databases, to show the students after they answered the data questions how to find the specific information they were referencing in their handouts.  These videos were also embedded on the blog post that I created for the project , allowing students to refer back to them after the class.

Finally, I created the questions for the class in Tophat, and put all questions inside their own folder within the Tophat Course for the class.  Because I was teaching three sections that day and wanted to keep the answers separate, I created two additional folders and copied the questions into those folders.

A picture of how I set up my folders in Tophat

Folder system in Tophat

The Delivery

At the start of the class session, I explained to the students that they would be doing the bulk of the heavy lifting in the research session.  The were instructed to find the handout, which was posted to the class Basecamp page, as well as log into the Tophat course.  Each team was required to have at least one laptop, but most tables had at least three.

Student research in action

The format for each database was as follows:

  1. Students answer Tophat question using data from the PDF handout.
  2. Chad show answers and talk about how the class did as a whole.
  3. Chad discuss how to properly answer the question or interpret the data correctly.
  4. Students answer another similar Tophat question about the database information.
  5. Chad hopes for improvement between step 2 & 4.
  6. Chad plays video to demonstrate how to find the information.

The slideshow below shows some of the questions used for the class.  Note that not all question had “correct” answers.

This slideshow requires JavaScript.

The Results

Not all questions had correct answers, as many of the answers were just their “interpretations” of the data and could not be judged right or wrong.  However, for the questions that did have specific correct answers, there was a noticeable improvement in the students’ ability to interpret the data correctly.    The two images below show just one example of how one class section immediately improved after I explained how to read the data correctly.  They really did get 49% smarter!

This slideshow requires JavaScript.

In the other two sections, 71% and 67% answered the first question correctly, immediately improving to 84% and 89% who answered correctly.  Seeing such substantial improvement across all three sections for the same question was very satisfying, while at the same time it was very cool to visibly show the classes that they had in fact learned something.

Other questions that did not have specific correct answers were very useful in showing the students how the data might be interpreted differently to tell a different story.  In many cases with consumer data, the story you tell and the answer you give depends on which data point you use, and to visually demonstrate how their classmates interpreted the data differently was effective in teaching them that there isn’t always one correct answer in business research.

In the days and weeks after the class session, I held approximately 30  consultations and answered another 25 questions about the project.  What is interesting is that in general, the questions I now received were about what available data might they best use to tell their story and where to find it, not about how to use the interface or how to interpret the data.  I spent less time last semester explaining the tools than I had in the past, allowing for more meaningful conversations with students about how they were using data to tell their story.

I clocked approximately 8 hours of preparation into this session, which might seem like a lot.  However, I should be able to use the exact same content for the sessions in future semesters, as the exercises were general enough to be applicable to future class projects.  The only thing I will change for future sessions is to create some questions to assess overall comfort/knowledge  for before and after the class session.  The sessions also required me to work a bit outside of my comfort zone in the classroom, and future sessions should improve with additional practice.

 

Teaching a one-shot library instruction session with TopHat

This spring I used Tophat to shake up the delivery of my large research sessions.  This is one example of how I have used Tophat to enhance my library research instruction.

The scenario
Each spring I am invited to give a one-shot, hour-long orientation to approximately 125 students who are part of the Global Consulting Program course.  The students take the semester-long course prior to their 3-week study-abroad trips where they will do consulting projects for real companies in China, France, Germany, Greece, Hungary, Spain, and Italy.  The goal of the 60-minute session is to give the students an overview — or in many cases, a refresher — on some of the tools they will use to conduct international business, company, cultural, and country research for their in-state class assignments and out-of-country consulting projects.
For the past few years, I would generally show up to class and just deliver a simple demo of some of the key resources that they would use in their projects.  Given that the class generally meets at 7 PM, I was lucky if less than 25 percent of the students fell asleep.
This spring I was approached by a new GCP program director, who invited me to do the orientation.  Since I had not worked with him before, I figured this was an ideal time to do something new.  We met and I told him what I had in mind, and he was very amenable to trying anything that would get the students more engaged.
 The setup

For this class, I have typically demonstrated resources right off my Best Research Strategies for Global Consulting page on my Business Blog.  I would continue to use this page for my new session, but wanted the class to do the bulk of the work themselves.  I  drafted some basic learning outcomes for the resources, and created nine questions that the students would answer, as teams, to push them to learn.  I put the  questions in TopHat, which I would use to present to the class and allow them to record their answers for all students to see.  Because the students are not enrolled in my TopHat course, I previously contacted TopHat to change my course to allow anonymous answers without the need to sign in (or enroll) to the course.   The professor also communicated with all students that they should bring their personal laptops to class.

The session

For the first five minutes of class, the Internet connection was painfully slow, and I struggled to log in to TopHat and bring up my class guide.  I thought my session, which I had spent about several hours preparing, was dead in the water.  However, the Internet finally behaved, and we were able to carry on as planned.

Screenshot of a typical Tophat discussion question

A sample TopHat question

I had each team work together to come up with a team name, since there were multiple teams going to each country.  I presented each question using the TopHat present mode, and allowed ample time for most groups to respond with their answers.  I selected the best answer with each question, and awarded the winning team for each question a goody bag.  The bag contained a sampling of library laptop stickers, pens, stress balls, and other assorted vendor junk that I had solicited from my colleagues (basically asked them to unload their junk and clean out their desks for me to give it to students).  The students got a kick out of the silliness of the prizes, and I thought the prizes stoked their competitive spirits.  After each question, I spent a short time explaining the answer correctly, and doing a short demo of the resource if necessary.

What I learned

Overall, I think the class went pretty smoothly.  I definitely think the students learned more, and were more engaged, than if I had simply stood in front of them and lectured for 45 minutes.  No one fell asleep.  I did have a few students who did not bring laptops, and if their neighbor also failed to bring a laptop, then those students pretty much checked out for the hour.  I appreciated that I could walk around the lecture hall and answer questions as they worked, allowing me to personally engage with some students in a way that would have been impossible in a traditional lecture format.

The professor provided great feedback and showed a great deal of enthusiasm for the class.  He even said, “From my own experience I know you had 5 minutes of work for every one minute of this class time, and it shows. This was fantastic.”  He was pretty much spot on, as I had about 3-4 hours in prep work for the class.   I think that the time spent was worth it, just in seeing the students do actual work and use the resources right away.   Given that the same class is offered every year and they usually go to the same countries, it was time well spent, as I can recycle the content and reuse the TopHat questions for future sessions.  This class also helped me set the groundwork for another class that I taught this semester, which I will be writing about soon.