I made my students 49% smarter and I can prove it

“Well looky there, you learned something!  You’re 49% smarter than you were 5 minutes ago!”  This aha! moment  occurred while teaching over 400 business students this fall.  Using Tophat in my business research instruction sessions, I was able to assess that my students did in fact learn something through my teaching.

The Challenge

Each semester I have the awesome opportunity to teach two research sessions to  over 400 sophomore business students.  The 400 students are divided into 3-4 sections, which I teach in the same day (it can be exhausting).  The first session is generally a typical 30-45 minute database demonstration, as they need to do a basic industry analysis for their first project.  For the project, all students are researching the same industry, so the tools they will need are pretty consistent and straight forward (Type an industry keyword in a search box, get some useful stuff).

The second project is a bit more challenging to teach to, as each 5-member student team can choose their own business to create.  The resources they need to successfully complete consumer demographic, local market, and competitor analyses are quite a bit more challenging to use than the sources for the first project.  The resources they need for the second project require significantly more creativity to use, as well as more brain power to interpret the data.  In the past when I have done a basic database demo of these resources for the second project, students were paying more attention to how to use the interface than they were in understanding how they might apply the data.  This was clearly demonstrated in the 20+ consultations that I held with student teams, as almost every team had questions about how to interpret the data.    The consultations were very repetitive, with each student team having the same questions.  This was not an efficient use of my time, and was certainly slowing student learning.  There had to be another way to teach them to use the data first, and the interface second.

The Setup

In order for the students to do thorough research for their projects, they really needed to deep dive into Simmons Oneview, SimplyAnalytics, and Bizminer.  I outlined my class sessions so that the students would first look at the data available from a single database and answer questions about the data.  This would be immediately followed by a demonstration of how to navigate and find the data in the specific database.

For the sessions to be relevant to their assignment,  I needed to make up a mock business concept that could adequately demonstrate how to interpret the data from the business databases.  I chose to investigate opening a store that would serve two of the three sports of mountain biking /road cycling, running, or golf.  I created a  data handout for class (pdf) with screenshots of demographic data and local market data that would be useful in researching my business concept.  I then drafted questions that would lead students to interpret the data to make decisions about which two sports my store should cater to, as well as the location of my store.  These questions were based on the types of questions they should be asking about their own business concept ideas.

I decided that a video tutorial of each database would be more efficient and consistent at demonstrating how to find the data in each individual database.  Simply pushing “play” and watching a video would help me to stay on track with my usage of class time by avoiding database/internet slowdowns and my own tangential ramblings, as might be the case during a live demonstration.  I created three videos, one for each of the essential databases, to show the students after they answered the data questions how to find the specific information they were referencing in their handouts.  These videos were also embedded on the blog post that I created for the project , allowing students to refer back to them after the class.

Finally, I created the questions for the class in Tophat, and put all questions inside their own folder within the Tophat Course for the class.  Because I was teaching three sections that day and wanted to keep the answers separate, I created two additional folders and copied the questions into those folders.

A picture of how I set up my folders in Tophat

Folder system in Tophat

The Delivery

At the start of the class session, I explained to the students that they would be doing the bulk of the heavy lifting in the research session.  The were instructed to find the handout, which was posted to the class Basecamp page, as well as log into the Tophat course.  Each team was required to have at least one laptop, but most tables had at least three.

Student research in action

The format for each database was as follows:

  1. Students answer Tophat question using data from the PDF handout.
  2. Chad show answers and talk about how the class did as a whole.
  3. Chad discuss how to properly answer the question or interpret the data correctly.
  4. Students answer another similar Tophat question about the database information.
  5. Chad hopes for improvement between step 2 & 4.
  6. Chad plays video to demonstrate how to find the information.

The slideshow below shows some of the questions used for the class.  Note that not all question had “correct” answers.

This slideshow requires JavaScript.

The Results

Not all questions had correct answers, as many of the answers were just their “interpretations” of the data and could not be judged right or wrong.  However, for the questions that did have specific correct answers, there was a noticeable improvement in the students’ ability to interpret the data correctly.    The two images below show just one example of how one class section immediately improved after I explained how to read the data correctly.  They really did get 49% smarter!

This slideshow requires JavaScript.

In the other two sections, 71% and 67% answered the first question correctly, immediately improving to 84% and 89% who answered correctly.  Seeing such substantial improvement across all three sections for the same question was very satisfying, while at the same time it was very cool to visibly show the classes that they had in fact learned something.

Other questions that did not have specific correct answers were very useful in showing the students how the data might be interpreted differently to tell a different story.  In many cases with consumer data, the story you tell and the answer you give depends on which data point you use, and to visually demonstrate how their classmates interpreted the data differently was effective in teaching them that there isn’t always one correct answer in business research.

In the days and weeks after the class session, I held approximately 30  consultations and answered another 25 questions about the project.  What is interesting is that in general, the questions I now received were about what available data might they best use to tell their story and where to find it, not about how to use the interface or how to interpret the data.  I spent less time last semester explaining the tools than I had in the past, allowing for more meaningful conversations with students about how they were using data to tell their story.

I clocked approximately 8 hours of preparation into this session, which might seem like a lot.  However, I should be able to use the exact same content for the sessions in future semesters, as the exercises were general enough to be applicable to future class projects.  The only thing I will change for future sessions is to create some questions to assess overall comfort/knowledge  for before and after the class session.  The sessions also required me to work a bit outside of my comfort zone in the classroom, and future sessions should improve with additional practice.

 

Teaching a one-shot library instruction session with TopHat

This spring I used Tophat to shake up the delivery of my large research sessions.  This is one example of how I have used Tophat to enhance my library research instruction.

The scenario
Each spring I am invited to give a one-shot, hour-long orientation to approximately 125 students who are part of the Global Consulting Program course.  The students take the semester-long course prior to their 3-week study-abroad trips where they will do consulting projects for real companies in China, France, Germany, Greece, Hungary, Spain, and Italy.  The goal of the 60-minute session is to give the students an overview — or in many cases, a refresher — on some of the tools they will use to conduct international business, company, cultural, and country research for their in-state class assignments and out-of-country consulting projects.
For the past few years, I would generally show up to class and just deliver a simple demo of some of the key resources that they would use in their projects.  Given that the class generally meets at 7 PM, I was lucky if less than 25 percent of the students fell asleep.
This spring I was approached by a new GCP program director, who invited me to do the orientation.  Since I had not worked with him before, I figured this was an ideal time to do something new.  We met and I told him what I had in mind, and he was very amenable to trying anything that would get the students more engaged.
 The setup

For this class, I have typically demonstrated resources right off my Best Research Strategies for Global Consulting page on my Business Blog.  I would continue to use this page for my new session, but wanted the class to do the bulk of the work themselves.  I  drafted some basic learning outcomes for the resources, and created nine questions that the students would answer, as teams, to push them to learn.  I put the  questions in TopHat, which I would use to present to the class and allow them to record their answers for all students to see.  Because the students are not enrolled in my TopHat course, I previously contacted TopHat to change my course to allow anonymous answers without the need to sign in (or enroll) to the course.   The professor also communicated with all students that they should bring their personal laptops to class.

The session

For the first five minutes of class, the Internet connection was painfully slow, and I struggled to log in to TopHat and bring up my class guide.  I thought my session, which I had spent about several hours preparing, was dead in the water.  However, the Internet finally behaved, and we were able to carry on as planned.

Screenshot of a typical Tophat discussion question

A sample TopHat question

I had each team work together to come up with a team name, since there were multiple teams going to each country.  I presented each question using the TopHat present mode, and allowed ample time for most groups to respond with their answers.  I selected the best answer with each question, and awarded the winning team for each question a goody bag.  The bag contained a sampling of library laptop stickers, pens, stress balls, and other assorted vendor junk that I had solicited from my colleagues (basically asked them to unload their junk and clean out their desks for me to give it to students).  The students got a kick out of the silliness of the prizes, and I thought the prizes stoked their competitive spirits.  After each question, I spent a short time explaining the answer correctly, and doing a short demo of the resource if necessary.

What I learned

Overall, I think the class went pretty smoothly.  I definitely think the students learned more, and were more engaged, than if I had simply stood in front of them and lectured for 45 minutes.  No one fell asleep.  I did have a few students who did not bring laptops, and if their neighbor also failed to bring a laptop, then those students pretty much checked out for the hour.  I appreciated that I could walk around the lecture hall and answer questions as they worked, allowing me to personally engage with some students in a way that would have been impossible in a traditional lecture format.

The professor provided great feedback and showed a great deal of enthusiasm for the class.  He even said, “From my own experience I know you had 5 minutes of work for every one minute of this class time, and it shows. This was fantastic.”  He was pretty much spot on, as I had about 3-4 hours in prep work for the class.   I think that the time spent was worth it, just in seeing the students do actual work and use the resources right away.   Given that the same class is offered every year and they usually go to the same countries, it was time well spent, as I can recycle the content and reuse the TopHat questions for future sessions.  This class also helped me set the groundwork for another class that I taught this semester, which I will be writing about soon.

My first experience in our new active learning classroom

Today was my first time teaching in our newly remodeled instruction space. It was one of the best teaching experiences I have had in my 14 years of library instruction.

mba_251

Our MBAs hard at work in Alden 251

The Room

The room is outfitted with three presentation screens at the front of the room.  There are six tables which seat 7, with a 48 inch television monitor at the head of each table.  The room is set up as BYOD (although we do have laptops available for checkout) and students can project via wireless from their laptops to the TV screens using Crestron Airmedia .

The Class Project

This is the first week of the fall semester, and the forty new MBAs have been off to a very fast start.  They will meet with their live client for their marketing project on Monday, so the timing was ideal that today (Friday) we were able to spend three hours together in our new classroom.

I held a similar research session for last year’s MBA class, but feedback from the students revealed it was too late to be of much use for their project (which they had just turned in).  Over the summer I was fortunate to work with our MBA coordinators to get a three hour block of time during their first week when their first project is launched.

The Class Activities

Last year’s session was held in a traditional campus classroom, with tables that held 6-8, with an instructor podium and projector.  I divided the group up into 8 teams, and gave each team 4 research questions.  The questions were geared toward their specific project (which I didn’t know they’d already completed) using specific resources.  Rather than stand at the front of the room and show the students how to use the particular resource, I designed the questions in a way that gave them enough information to allow team exploration of a resource on their own.  Each question listed the name of the database, and offered enough context clues to give the students some hints on what to look for.    After spending 45-60 minutes working through the questions, I then had each team report on how they solved the question, while also demonstrating the resource on the computer and projector for the rest of the class.  Many students liked this approach, though equally as many would have preferred a more traditional “database demonstration” approach.    Unfortunately, since I assigned each group their four questions all at once, many of the teams farmed out the questions to individuals, rather than work on each question as a team.    I’ve since used this “team-based exploration and teach others model” a few more times in other classes (mostly with graduate students) with varying levels of success.

For today’s session, I built upon the same explore/learn/show model but incorporated a few changes.  I was limited to 6 teams of 6-7 students (which was a touch large) since we only have six tables in our new room.  Rather than  give the students a handout of all the questions (and tell each team to answer different questions), I gave all groups the same questions.  Also, instead of using a handout, I put each question in a Tophat course that I developed for the class.  Each question was presented to the students individually, ensuring that the entire team was limited to working on the same question.  Each question was presented as a discussion question in Tophat, and I asked that each team answer with  [Team Name:  Answer]  format.  This allowed me to track team answers and time spent on the questions, while also identifying stellar responses to showcase to the rest of the class.  (Here are the questions if you would like to use them for your own business instruction class, or adapt them for another subject.)

The Crestron control station at the front of the room enables the instructor to grab and show a screen of one of the six table monitors.  I would use Tophat to identify an interesting answer (or a team that had yet to share) and then use the Crestron to present that team’s screen on the three large monitors at the front of the room.  I would then ask that team to explain to the other students how to navigate the database and how they found their answers.  By doing this, I was never at the front of the room doing a demonstration, but rather was coaching a student through the resource, asking questions, and prompting them to an answer.  One key takeaway about letting the students drive is that I always learn how to use our resources in a different way.

In Conclusion

I spent about 4 hours preparing for this class, which included writing the questions in Tophat, learning how to use it,  and running through the resources to make sure the questions made sense.  I also practiced for 30 minutes with the technology in the room.   With the exception of one Crestron monitor failing to connect, all of the technology worked flawlessly.     The students seemed to like the room, and appreciated getting out of their traditional classroom space.  No one fell asleep, and it appeared that most teams had participation from everyone at the table.

My last three Tophat questions were meant to poll how the students felt about the research session, what they had learned, and how they might improve the session.  I only had one student who said they would have preferred a more traditional demonstration, and many students said the session could have been improved with donuts and food (I did give them a 10-minute break to go to the coffee shop in Alden).    Many felt tired after the three hours, and as their instructor, I definitely was exhausted. Most seemed to feel inspired and confident that they had a good enough grasp of the resources to do well on their marketing project, and I was pleased to see groups of students working together, teaching each other, and doing relevant research for their project.

My favorite piece of feedback was a great finale to the class, this week, and this post.  The student posted:

“Chad knows his stuff. What a guy. I might love him.”

 

YouTube and Fox News for an evaluation exercise

I’ve been doing an evaluation exercise with many of my English 151 classes for the last couple of years.  It works pretty well, and I’ve had several instructors request that I repeat it again for their future classes.  Many of my colleagues have been asked about this exercise, so I thought I would put it down it writing.

First, a little background on our English 151 library sessions.  We generally get to see each section of English 151 for a 2-3 hour period, usually during the middle of the 10-week quarter.  We teach them how to use the online catalog, how to search for articles, a little about navigating our website, and usually a little about evaluation.  The English classes are our best bet in getting most of the first-year students familiar with the library.  Many students won’t ever have another library instruction session in their college career, but rather than try to teach them everything they could possibly know about the library, we try to address more general themes.  One of those themes is evaluating sources, and we all tend to teach this a little differently.  Here is one of the more successful exercises that I use to teach evaluation.  If you have any questions or if anything isn’t clear, please post a comment and I’ll try to answer as best as I can.

The Exercise

1.  I play the following YouTube video for the class. It takes about seven minutes.

2.  While the video is playing, I take notes on a whiteboard at the front of the class.  Basically I am writing down things the people say in the video that really stick out at me, things that I might want to question.  If you listen to the video, you’ll get the idea of the type of things that might raise a flag.

Notes on the whiteboard

3.  After the video is over, I ask the class what they thought of the video, and this usually generates some conversation.  If no one talks, I’ve got the notes on the whiteboard at the front of the class.  I’ll mention things like the “Luke Skywalker meets…..” comment, or ask them about what things the panel may have said that may cause them to be less credible.   I also ask them about the supposed “expert” on the panel, and ask what makes her credible.

4.  With the expert, I like to point out that she had not played the game that she was criticizing at all.  I also mention that the anchorwoman did not allow the real expert, the game journalist, to speak much at all.  We then talk about all the other panelists, and discuss what gives them the authority to talk about the game.  As an example, the anchorwoman “went on the internet” to do research, one panelist questioned “what happened to the days of Atari and PacMan” and that she felt “old”, and another panelist compared games to Playboy Magazine.

5.  In examining the clip, I like to specifically focus on one of the expert’s points about game statistics.  She alleges that teenagers are the majority of game players.  However, if you check the Entertainment Software Association’s website, you’ll see that the average gamer is actually 35 years old.  The expert also mentions a “new study out of the University of Maryland,” but I try to point out that there are numerous studies about video games, and nearly all of them contradict each other.  If you want to find a study that says video games make you kill people, then you can find that pretty easily.  If you want to find a study that says video games will make you smarter and a better team player, entire books have been written on those topics as well.  The point is to look critically at these studies and find other information that strengthens and weakens the argument.

6.  I then point to this blog post on Joystiq about the Fox News/Mass Effect controversy.  I explain to them what happened in a nutshell.  First, Fox News aired the video shown above, then someone (most likely an angry gamer) copied the video and uploaded it to YouTube.  (FYI, more than one video was posted). Next, multiple gaming blogs embedded the video(s) on their pages, whereby millions of gamers saw the video and became enraged.  Not content to just sit on their hands, many of these gamers went to Amazon.com and rated the expert’s book (which happened to be promoted during the video segment on Fox).  Amazon actually removed over 400 negative reviews of the book, but most of them said something like  <paraphrase>”after never actually reading the book, I can tell you that this book is an utter piece of garbage and the author has no idea what she is talking about.”  I can say this because, since I haven’t read the book, I am still an authority on the subject.”</paraphrase>   The expert later apologized publicly via the New York Times, but I don’t think Fox News ever ran a retraction of the story.

7.  The entire exercise, including watching the video and discussion, generally takes no longer than 15 minutes.  More than 15 minutes, and I find myself belaboring the point too much and their minds start to drift.  If you use this, try to keep it fresh and lively, engage the students for a few minutes, then move on to the next thing on your agenda.  I usually do the exercise after they we have had time for hands-on work and they have found a few articles or books.  It’s a good way to bring the class focus back together after the hands-on individual activities.

So what’s the point?

The point of the exercise is to demonstrate to the students that evaluation of information goes beyond  telling the difference between popular and  scholarly articles.  This exercise shows them that they should look at things critically, regardless of whether they are doing academic research, watching the news, buying a new camera, or trying to decide which movie to go see.  Even in real life outside of academia, we are required to make choices about the information that we ingest and digest.  Even when information is fed to us via Fox News, CNN, the New York Times, our professors, or our mothers,  it’s important to understand and look for bias and misinformation.  I get better with the exercise the more I do it, and by the conversation it starts with the students, I believe it really is effective at making my point.

I hope that this is useful to someone and that if people use it, they’ll make it better.  If you try it and it works, please let me know.  I’d love to hear what worked, what didn’t, and what changes you made to make the exercise better.

I’m also curious what other librarians and teachers are doing, as I’m always looking for new ways to be more effective in the classroom.  Have you seen something that really worked, or have you tried something that really made the point of evaluation clear to the students?   If so, I’d love to hear about it.

How to present like Steve Jobs

I’m not an Apple fanboy by any means, but I will say that Steve Jobs does have a knack for drumming up a little excitement. This video shows how you can incorporate some of Jobs’ style into your own presentations.

Granted we don’t have a product as exciting as in iPhone when we teach students how to do library research, but we can offer such things as “free information” that can “make their lives easier” and get them “well on their way to completing their research” with “as little effort as possible.”