This article is more than 5 years old.
On Monday, May 21, Hu Womack, Kevin Gilbertson, Lauren Pressley, and I drove to the High Country of Boone to attend NC-LITe. It was doubly wonderful for me because I had the opportunity to travel with three amazing colleagues to one of my favorite places on earth.
It seems hard to take in that it has been seven years since I left my position at Appalachian as an Instruction Librarian. Our family moved from Boone three months before the Library moved into their new facility so it was fun just seeing the building! When we drove up in the parking deck and told the attendant that we were there for NC-LITe, she proclaimed in her authentic mountain brogue, “We finally got us one!” and for me, it felt like I was home.
When we entered the library, Scott Rice greeted us and escorted us to the fourth floor conference room. We were welcomed with a beautiful spread of pastries and coffee. Institutions represented were Salem College, UNC-G, High Point University, Appalachian, and WFU. The intimate group encouraged informal discussion.
The day began with around the table introductions and catch-up. Most of the shared information had to do with strategies for reaching the masses by offering consistent content for specific courses such as Expository Writing and Freshmen Seminars. (We are so fortunate at Wake to be able to tailor our Freshmen Seminars to the content taught.) It was interesting to note that each represented institution is currently using or has plans to use a discovery tool similar to Summon.
After the round table discussions, Kevin, Lauren and I presented our findings from our Summon usability study. Kevin began the discussion by offering some historical perspective regarding how our Library first used Multi-search and then implemented Summon last summer. Our tech team was forced to move quickly in order to get it up and going before Erik Mitchell left. In light of this, the tech team decided it would be particularly helpful to conduct a usability study to see how ZSR’s Summon (the “Everything” link) was being used.
After Kevin’s introduction, we used a clicker presentation to introduce our findings. (Note: having three techies on this trip was a true gift!!) Our usability study was based loosely on NC State’s usability study which was done three years ago and was sponsored by Summon (NC State was a Summon beta site). Their study was led by an outside consultant and their report can be found here: http://www.lib.ncsu.edu/userstudies/studies/2010_summon/
For our study, we used eight student workers who were available for the afternoons that we conducted the usability study (thank you, ZSR student supervisors for sharing your students with us!). Each individual interview lasted 30-45 minutes and we met in the vacant office of the Instruction Suite (302B). Kevin conducted the interviews while Lauren and I took notes.
We began with what NC State called “Standard Framing” which helps to determine the student’s proficiency at navigating the Library website. This ended up being a very interesting section of the study! Students were asked to go to the Library homepage and asked to find a book about Bill Gates but not by Bill Gates and to tell where it was located in our library. A couple of students thought they had a book about Bill Gates when it was about the Bill and Melinda Gates Foundation. A couple of students did not notice that they had selected an ebook, and at the end of the interview one student asked, “What is an ebook?” One student found a book in the Professional Center Library and did not realize that it was not in the ZSR Library.
Students were given a journal article citation and were asked to locate the article. There were two students who used the “Journals” link to search for the article title and those students could not locate the article (one of those eventually used the “Everything” link and was successful). Two students used the journal names in the journal link and they were successful. One used the “Article” link and she was successful, and one started with the “Everything” link and was successful. One student started with the catalog link and after a long and winding search, located the article.
We found similar results when we asked students to find a scholarly journal article about stress among college students. One student started with the Databases link and another started with the Journals link and each typed in their subjects within the boxes; neither was successful. Students who started with the “Everything” link or the “Articles” link were successful. One student (not the first Database selector) got frustrated with the Databases link and decided to switch to JSTOR because her professor showed it to her once and she knew it was really big.
We then switched to having the students search only using the Everything link (Summon). We asked them to find a book by an author they liked. Two of the students were not successful with their search because their authors were J.D. Salinger and H.G. Wells; many sources came up about these authors, but not by these authors (to be successful, they should have used the Books/eBooks facet). One student searched for “Harry Potter” and there were six entries before a “Harry Potter” book appeared on the list.
Students were asked to find the latest issue of the Journal of the American Chemical Society. None of the students were successful with this search. What happened was that they began with a searches for the [Journal of the American Chemical Society] which produced a list, but the first three entries on that page directed them to Carpenter Library. (To correctly answer this question, they needed to click the fourth entry to see our subscription to the latest articles.) When they were not successful with the first couple of links, they switched to the “journal articles” facet where they lost the ability to sort by relevance. Summon does an excellent job searching by title, but after that, relevance is lost. Several of the students were confident that they had found the answer, when in fact they were looking at articles where the Journal of the American Chemical Society was cited.
We asked the students find to an article from the New York Times from the month and year they were born. Only two students noticed that newspapers were excluded from the Summon search results (it could be turned on from the facets). Two students were simply not successful with this search, two other students thought they were successful but their articles were from New York Times Magazine, and two students were successful because they ended up with links to Proquest Historical Newspapers.
None of the students were successful finding an article on “Is genetically modified “golden rice” safe?” As the interviewers, we really wished the student would use quotation marks to look for strings of words or “Ctrl-F” to search for words in their results. We hope that our LIB100 instructors will start teaching these skills as part of their curriculum!
We asked the students if they would recommend the Everything link to their friends, and five out of eight said they would “if they don’t know where else to start.”
We also asked the students what they would change about the Everything link and they had a few suggestions. The number one suggestion was to make it so that it could do a better job searching by relevance; they worded this concept something like this: If you search for “Harry Potter,” a “Harry Potter” book should come up to the top. A couple of students said that they wished that you could limit the category options before you do your search (searching by author, title, etc.). Another student said that it would help if the results were sorted by format with maybe the books on top followed by the journals.
After our presentation, Margaret Gregor, Appalachian’s Instructional Materials Center Librarian and Scott Rice, Appalachian’s E-Learning Librarian, presented a tool they created to provide virtual tours of the IMC. The students were required to find 16 hidden puzzle pieces and a book in the collection (the students were instructed to write down the bar code of the book they found). Scott set up a temporary password for us so that we could try it out. They reported that over 500 students had completed the tour. Here is a link to this site and I can give you a password that you can use if you are interested: http://library.appstate.edu/elearn/imc
After Margaret and Scott’s discussion, we then had some group discussions about things we were doing at different institutions. That discussion was followed by lunch at “Our Daily Bread” in downtown Boone and then a tour of the Library.
Overall, it was a great way to spend the day! I’m already looking forward to NC-LITe in the fall!!
2 Comments on ‘NC-LITe and WFU Summon Usability Study Findings Presented at Appalachian State University’
It was a great day! And it was nice to talk about our Summon findings, and even more so after the trip to Duke the week before! I was most struck that the usability study both gave ideas for how to improve the service (as usability studies are designed to do) but also to help us think about how and what we teach (a nice bonus benefit). I’m looking forward to seeing improvement to Summon and our teaching based on the findings!!
Such great info on how students use our resources! Thanks for the summary – I’m already plotting how to improve how I teach Summon in my one-shots and LIB210 classes.