Technical documentation is boring: It's boring to create, and it's especially boring to read. Unfortunately, technical documentation is also incredibly important for users and developers of digital libraries to understand their functionality. This is especially true of large scale digital libraries that provide application programming interfaces (APIs) for accessing large, heterogeneous collections of digital artifacts. In this paper we draw on the concept of assesability design in order to evaluate the coverage and completeness of technical documentation for five large-scale digtal library APIs. here describe goal of work, results and implications of work once you better understnad
It may seem trite to observe the profound effect that large-scale digital library initiatives have had on the access, and use of of cultural heritage artifacts; but, it shouldn't. Initiatives such as Europeana, the Hathi Trust, and the Digital Public Library of America (DPLA) have created collections of diverse digital materials that make new forms of scholarship, and new modes of cultural understanding possible. Perhaps most profoundly, these new aggregations of resources are allowing the digital humanities to flourish - realizing the potential of not only a 'Renaissance' view of great ideas, but also a pragmatic view of ourselves and the role of the arts in our public lives (Adams, 2014). Consider the following examples:
-
In his revelatory economic work, Thomas Piketty claimed that the 19th and 20th century novel was a societal mirror - the absence of financial verbiage reflected - albeit subtly- recognition of income inequality. Underwood et al. used the HathiTrust digital library - a collection of X million texts - to show that in fact, fiscal language became more pronounced during this time period (2014).
-
The Culture-omics work at Harvard + Internet Archive
-
Lev Manovich's work at CUNY + Archives of Europeana and Flickr
These examples are not subtle shifts in scholarship - they represent paradigmatic new ways of knowing ... and they demonstrate a need for new approaches to traditional concepts in digital library development, such as the scholarly collections (Wicket et al, 2014), accessibility (Gonclaves et al., 2007), search (Willis and Efron, 2012) intellectual property rights (Downie et al., 2012), user literacy (Forte et al, 2014), and appraisal (Feinberg, 2013).
In this paper, we focus on an overlooked but profoundly important ascpect of digital library development - technical documentation - focusing on the assesability of different types of documenation available to users of application programming interfaces (APIs).
[Make argument from standards lit ... and also draw on tech. documentation lit ]
For instance, omplete and timely technical documentation has been show to be the most important factor in establishing trust between developers and users of open-source software (Dagenais & Robillar, 2010).
The design of technical documentation is generally described in terms of an economic benefit - it makes work more efficient - saving time, money, and effort.
There is a long history of studying technical documentation in terms of its understandability, effectiveness, and ultimately it's financial impact on organizations. As early as the 1970s economic studies of technical documentation were being conducted - finding that improved technical documentation could save the government $1.7 million (Shriver and Hart, 1975).
A distinction can be made between the types of technical documentation we've described thus far:
-
developer documentaiton [write defintion of typical tech. documentaiton here here]
-
communnity documentation, such as wikis used by open source projects, allow anyone involved with the creation or use of a resource to contribute to documentation about the resource.
-
crowd documentation, such as developer blogs and Q and A, are individual contributions that often focus on documenting a single aspect of the resource. In aggregate, crowd documentation has proven to be rich in both coverage and quality [2011]
Q & A sites, development blogs, and social media also play an important role in supplementing existing documentation - in some cases - offering high levels of coverage and author engagement (Parnin and Treude, 2011).
Our focus is on all three types of documentation for application programming interfaces (APIs) in digital libraries.
[ Brief discussion of what APIs are and why they matter to DLs]
APIs are thus an important aspect of end-user interactions with a digital library. API's, and technical documentation more generally, are not part of the evaluation digital library development. This paper seeks to fill this gap in two ways:
- Provide a background to API usability, documentation, and evaluation from software industry.
- Evaluate the assesability of technical documentation for the APIs in five large-scale digital libraries ( [note - assesability of different types of documentaiton is based on coverage and completeness - criteria hinted at below]
[Wiki has start of lit review]
- In a field study of 440 software engineers, R + D find that the biggest obstacle to learning APIs is poor documentation and insufficient learning resources. They recommend the following criteria for improving documentation: documentation of intent; code examples; matching APIs with scenarios; penetrability of the API; and format and presentation. (Robillard and Deline, 2011)
these recommendations can be used as evaluation criteria for completeness
Tools for creating API documentation
- Jungloids,
- Jadeite, and
- JavaDOC
[Write formal research questions - thinking they will correspond to ]
RQ 1. Coverage and Completeness of Docoumentation.
- RQ 1.1 Coverage is using survey of methods found in "community documentaiton " and "crowd documentaiton" similar to approach used by Parnin and Treude, 2011 (see table 1)
Note about RQ 1.1 - may be impossible because documentation is so poor
- RQ 1.2 Completeness is degree to which "developer documentation" conforms to criteria outlined by Robillard and Deline, 2011a, and 2011b (see table 2)
Table 1
| DL | Link | Number of Methods |
|---|---|---|
| DPLA | http://dp.la/info/developers/codex/' | no methods |
| Hathi Trust | http://www.hathitrust.org/data_api | no methods |
| Europeana | http://pro.europeana.eu/api | no methods |
| NYPL Digital Collections | http://api.repo.nypl.org/ | no methods |
| DDB | https://www.deutsche-digitale-bibliothek.de/content/terms/api/ | no methods |
| arXiv | http://arxiv.org/help/api/index | no methods |
Table 2
| - | Documentation of intent | Code examples | Matching APIs with scenarios | Penetrability of the API | Format and presentation |
|---|---|---|---|---|---|
| DPLA | |||||
| HathiTrust | |||||
| Europeana | |||||
| NYPL Digital Collections | |||||
| DDB | |||||
| arXiv |
RQ 1.1 None of these API docs describe methods <--- Fuck this noise RQ 1.2 [Need to better understand what criteria of assesment are in Robillard and Deline]
-
B.Dagenais and M. P. Robillard. Creating and evolving developer documentation: Understanding the decisions of open source contributors. In Proceedings of the 18th ACM SIGSOFT International Symposium on the Foundations of Software Engineering, pages 127–136, November 2010
-
Robillard and Deline 2011a http://dl.acm.org/citation.cfm?id=2036677
-
Robillard and Deline 2011b -> http://www.cs.mcgill.ca/~martin/papers/cr2014a.pdf
-
Wickett, K. M., Isaac, A., Doerr, M., Fenlon, K., Meghini, C., & Palmer, C. L. (2014). Representing Cultural Collections in Digital Aggregation and Exchange Environments. D-Lib Magazine, 20(5), 2.
-
On Crowd Documentaiton http://neverworkintheory.org/2012/06/05/crowd-documentation.html
-
On usability of APIs http://neverworkintheory.org/2011/11/08/a-field-study-of-api-learning-obstacles.html
-
On (super cool) evolution of docummentation http://neverworkintheory.org/2011/08/23/creating-and-evolving-developer-documentation.html
-
D. Mandelin, L. Xu, R. Bod ́ık, and D. Kimelman. Jungloid mining: helping to navigate the api jungle. In Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation, PLDI ’05, pages 48–61. ACM, 2005
-
J. Stylos, B. A. Myers, and Z. Yang. Jadeite: improving api documentation using usage information. In Proceedings of the 27th international conference extended abstracts on Human factors in computing systems, CHI ’09, pages 4429–4434, New York, NY, USA, 2009. ACM
-
Shriver, E. and Hart, F. Study and proposal for the improvement of military technical information transfer methods. Aberdeen Proving Grounds, MD: U.S. Army Human Engineering Laboratory, December 1975
-
http://www.neh.gov/about/chairman/speeches/address-national-federation-state-councils
Other things to look at:
-
Metadata Coverage Index: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3558968/
-
Gonçalves, M. A., Moreira, B. L., Fox, E. A., & Watson, L. T. (2007). “What is a good digital library?”–A quality model for digital libraries. Information Processing & Management, 43(5), 1416-1437.
-
Downie, J. S., B. Plale, M. S. Poole, and R. H. McDonald, Toward non-consumptive formal evaluation challenges using the HathiTrust Research Center digital collections, Japanese Association for Digital Humanities Conference (JADH 2012), http://www.jadh.org/jadh2012, Tokyo, Japan, Sep 2012.
-
Feinberg, M. (2013, July). Comparative appraisal: systematic assessment of expressive qualities. In Proceedings of the 13th ACM/IEEE-CS joint conference on Digital libraries (pp. 115-124). ACM.
-
Forte, A., Andalibi, N., Park, T., & Willever-Farr, H. (2014, April). Designing information savvy societies: an introduction to assessability. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems (pp. 2471-2480). ACM.