Follow and join in this discussion by using your free NINES account.
Describing the Digital Project 
Posted by Dana Wheeles on Sep 22, 2009 03:54PM
When the NINES Exhibit Builder was fully released in Fall of 2008, it seemed that it was uniquely suited to making guided tours of NINES' partner sites. One of the first exhibits made was the guide to Romantic resources, the making of which prompted an interesting question: how does one describe a digital resource? Unlike a book, which can be held, examined in terms of size and weight, and quickly perused by its table of contents or index, digital projects present the user with an ostensibly opaque object. There is only so much information one can glean from a home page (even the best designed home page!). Random searches or browsing cannot give you the sense of the extent of the resource, only its basic structure.
If we are to peer review resources, how do we describe them - to the editorial boards, to the reviewers, to the users? What kinds of metrics are necessary for 'pre-reading' a digital resource?
If we are to peer review resources, how do we describe them - to the editorial boards, to the reviewers, to the users? What kinds of metrics are necessary for 'pre-reading' a digital resource?
This comment was modified on Jan 11, 2010 02:39PM
This post is protected by a Attribution Non-Commercial Share Alike Creative Commons License. Learn more here.Replies to this topic (1)
Posted by Kristin Jensen on Sep 23, 2009 01:04AM
Here are a few different approaches to description that I can think of:
1. Functional, in terms of what the user can DO with the digital resource. This idea comes partly from the usability testing that I've participated in at UVa's usability lab. Usually the first question they ask is, "From this web page, what do you think you can do?" Rather than stopping at describing the contents of a digital resource, we might describe what it allows the user to do. For example, rather than merely saying that a site hosts electronic texts, we might say that it allows the user to view and read the texts, search the texts by exact character string or by "fuzzy" search, view manuscript or first edition page scans side by side with e-text transcriptions, download PDF or e-reader versions of the texts, collate successive editions of certain texts, stream or dowload audio recordings of the texts being read aloud, collect and tag texts for later reference . . . or whatever it is that the site allows.
Dana raised questions about both description and evaluation (peer review). I think when it comes to evaluation of sites that collect or re-present existing resources in some way (as opposed to, for example, digitally published journals), it's especially important to ask "what does this resource allow us to do that we couldn't do before?"
2. Quantitative. To a certain extent, we can describe a book in quantitative terms (how many pages? how many chapters? how many references in the bibliography?). There are ways of describing digital media in quantitative terms. We can say how many texts or images a site hosts, or how many MB of data, or how many external links. Some sites brag about such factoids on their home pages. In other cases, we'd have to get in touch with the site's maintainers and ask them. In either case, the accuracy of such quantitative claims could be difficult to verify, especially if a site hosts a very extensive collection. There are also some pretty obvious limitations on the usefulness and relevance of this kind of quantitative description, especially when you're comparing (for example) GIS apples to poetry database oranges.
3. Traffic / use. One dimension for describing an online resource would be noting the number of visitors / users / registered users / pages served / searches executed / blog posts / discussion board messages / etc. A more nuanced level of description could try to give an account of the most frequented paths through the site, outside sources that refer to the site, networks of users on the site, etc. This kind of description would obviously require the cooperation of the site's maintainers.
4. Trustworthiness. One function of peer review is to guarantee the trustworthiness of academic resources. This could be incorporated into the description of digital resources; a lot of the same metrics of trustworthiness that we use for academic works in print media would be applicable to digital resources. If a site reproduces primary materials, what is its source for those materials? (e.g. which edition of a book?) Is it a reliable source? If a digital resource makes factual assertions or cites existing works of scholarship, those references can be fact-checked the same way a journal article appearing in print can be fact-checked. Who is sponsoring the creation of the resource in question? Do they have any distorting biases? Who is executing the work? Do they have credentials / background that gives us confidence in their expertise?
1. Functional, in terms of what the user can DO with the digital resource. This idea comes partly from the usability testing that I've participated in at UVa's usability lab. Usually the first question they ask is, "From this web page, what do you think you can do?" Rather than stopping at describing the contents of a digital resource, we might describe what it allows the user to do. For example, rather than merely saying that a site hosts electronic texts, we might say that it allows the user to view and read the texts, search the texts by exact character string or by "fuzzy" search, view manuscript or first edition page scans side by side with e-text transcriptions, download PDF or e-reader versions of the texts, collate successive editions of certain texts, stream or dowload audio recordings of the texts being read aloud, collect and tag texts for later reference . . . or whatever it is that the site allows.
Dana raised questions about both description and evaluation (peer review). I think when it comes to evaluation of sites that collect or re-present existing resources in some way (as opposed to, for example, digitally published journals), it's especially important to ask "what does this resource allow us to do that we couldn't do before?"
2. Quantitative. To a certain extent, we can describe a book in quantitative terms (how many pages? how many chapters? how many references in the bibliography?). There are ways of describing digital media in quantitative terms. We can say how many texts or images a site hosts, or how many MB of data, or how many external links. Some sites brag about such factoids on their home pages. In other cases, we'd have to get in touch with the site's maintainers and ask them. In either case, the accuracy of such quantitative claims could be difficult to verify, especially if a site hosts a very extensive collection. There are also some pretty obvious limitations on the usefulness and relevance of this kind of quantitative description, especially when you're comparing (for example) GIS apples to poetry database oranges.
3. Traffic / use. One dimension for describing an online resource would be noting the number of visitors / users / registered users / pages served / searches executed / blog posts / discussion board messages / etc. A more nuanced level of description could try to give an account of the most frequented paths through the site, outside sources that refer to the site, networks of users on the site, etc. This kind of description would obviously require the cooperation of the site's maintainers.
4. Trustworthiness. One function of peer review is to guarantee the trustworthiness of academic resources. This could be incorporated into the description of digital resources; a lot of the same metrics of trustworthiness that we use for academic works in print media would be applicable to digital resources. If a site reproduces primary materials, what is its source for those materials? (e.g. which edition of a book?) Is it a reliable source? If a digital resource makes factual assertions or cites existing works of scholarship, those references can be fact-checked the same way a journal article appearing in print can be fact-checked. Who is sponsoring the creation of the resource in question? Do they have any distorting biases? Who is executing the work? Do they have credentials / background that gives us confidence in their expertise?