Code4Lib - User contributions [en]

Libraries Sharing Code

2013-02-18T21:09:11Z

Tdonohue: /* Institutions on GitHub */

A number of libraries have organizational repositories in GitHub. These can be very valuable and we attempt to collect them here.

==Institutions on GitHub==
* [https://github.com/gwu-libraries/ George Washington University Libraries]
* [https://github.com/gvsulib Grand Valley State University Libraries]
* [https://github.com/nypl/ The New York Public Library]
* [https://github.com/NYULibraries NYU Libraries]
* [https://github.com/psu-stewardship Penn State Digital Stewardship]
* [https://github.com/organizations/RockefellerArchiveCenter Rockefeller Archive Center]
* [https://github.com/adsabs/ SAO/NASA Astrophysics Data System]
* [https://github.com/ucsdlib?tab=repositories UCSD Library]
* [https://github.com/ualibraries The University of Arizona Libraries]
* [https://github.com/ui-libraries University of Iowa Libraries]
* [https://github.com/UNC-Libraries University of North Carolina at Chapel Hill Libraries]
* [https://github.com/ndlib University of Notre Dame Hesburgh Libraries] (And [https://github.com/ndlibersa the CORAL stuff])
* [https://github.com/yalemssa Manuscripts and Archives, Yale University Library]
* [https://github.com/yorkulibraries York University Libraries]
* [https://github.com/chattlibrary Chattanooga Public Library] (currently empty)
* [https://github.com/DarienLibrary Darien (CT) Library] (currently empty)
* [https://github.com/NCSU-Libraries NCSU Libraries] (currently empty, but see individuals below)
* [https://github.com/pulibrary Princeton University Library]
* [https://github.com/osulibraries/ Ohio State University Libraries]

==Institutions with Non-GitHub open repositories==
* University of Florida: SobekCM software [http://sourceforge.net/directory/?q=sobekcm Sourceforge], [http://code.google.com/p/sobekcm/ Google code], [http://ufdc.ufl.edu/software UFDC institutional site]
* [http://eXtensibleCatalog.org eXtensible Catalog] repositories: [http://code.google.com/p/xcmetadataservicestoolkit/ Metadata Services Toolkit], [http://code.google.com/p/xcoaitoolkit/ OAI Toolkit], [http://code.google.com/p/xcncip2toolkit/ NCIP Toolkit] (all these on Google code), [http://drupal.org/project/xc Drupal Toolkit] (on drupal.org)

==Other Library Organizations on GitHub==
* [https://github.com/projectblacklight Project Blacklight]
* [https://github.com/projecthydra Project Hydra]
* [https://github.com/islandora Islandora]
* [https://github.com/artefactual Artefactual]
* [https://github.com/duraspace DuraSpace], along with projects: [https://github.com/dspace DSpace], [https://github.com/fcrepo/ Fedora], and [https://svn.duraspace.org/view/duracloud/ DuraCloud (not yet on GitHub)]

==Individuals on GitHub==
* [https://github.com/jronallo Jason Ronallo] North Carolina State University
* [https://github.com/cazzerson Jason Casden] North Carolina State University

Libraries Sharing Code

2013-02-18T20:59:06Z

Tdonohue: /* Other Library Organizations on GitHub */

A number of libraries have organizational repositories in GitHub. These can be very valuable and we attempt to collect them here.

==Institutions on GitHub==
* [https://github.com/gwu-libraries/ George Washington University Libraries]
* [https://github.com/gvsulib Grand Valley State University Libraries]
* [https://github.com/nypl/ The New York Public Library]
* [https://github.com/NYULibraries NYU Libraries]
* [https://github.com/psu-stewardship Penn State Digital Stewardship]
* [https://github.com/organizations/RockefellerArchiveCenter Rockefeller Archive Center]
* [https://github.com/adsabs/ SAO/NASA Astrophysics Data System]
* [https://github.com/ucsdlib?tab=repositories UCSD Library]
* [https://github.com/ualibraries The University of Arizona Libraries]
* [https://github.com/ui-libraries University of Iowa Libraries]
* [https://github.com/UNC-Libraries University of North Carolina at Chapel Hill Libraries]
* [https://github.com/ndlib University of Notre Dame Hesburgh Libraries] (And [https://github.com/ndlibersa the CORAL stuff])
* [https://github.com/yalemssa Manuscripts and Archives, Yale University Library]
* [https://github.com/yorkulibraries York University Libraries]
* [https://github.com/chattlibrary Chattanooga Public Library] (currently empty)
* [https://github.com/DarienLibrary Darien (CT) Library] (currently empty)
* [https://github.com/NCSU-Libraries NCSU Libraries] (currently empty, but see individuals below)
* [https://github.com/pulibrary Princeton University Library]

==Institutions with Non-GitHub open repositories==
* University of Florida: SobekCM software [http://sourceforge.net/directory/?q=sobekcm Sourceforge], [http://code.google.com/p/sobekcm/ Google code], [http://ufdc.ufl.edu/software UFDC institutional site]
* [http://eXtensibleCatalog.org eXtensible Catalog] repositories: [http://code.google.com/p/xcmetadataservicestoolkit/ Metadata Services Toolkit], [http://code.google.com/p/xcoaitoolkit/ OAI Toolkit], [http://code.google.com/p/xcncip2toolkit/ NCIP Toolkit] (all these on Google code), [http://drupal.org/project/xc Drupal Toolkit] (on drupal.org)

==Other Library Organizations on GitHub==
* [https://github.com/projectblacklight Project Blacklight]
* [https://github.com/projecthydra Project Hydra]
* [https://github.com/islandora Islandora]
* [https://github.com/artefactual Artefactual]
* [https://github.com/duraspace DuraSpace], along with projects: [https://github.com/dspace DSpace], [https://github.com/fcrepo/ Fedora], and [https://svn.duraspace.org/view/duracloud/ DuraCloud (not yet on GitHub)]

==Individuals on GitHub==
* [https://github.com/jronallo Jason Ronallo] North Carolina State University
* [https://github.com/cazzerson Jason Casden] North Carolina State University

Mentorship

2012-06-04T18:06:52Z

Tdonohue:

We're looking to try out a mentorship program through the #code4lib channel on IRC Freenode. The idea is that experienced people who are willing to volunteer some time would provide 1-on-1 mentoring to those who would like to learn during an "Office hours" period, once a week. The idea is sort-of modelled on the Drupal project's [http://drupal.org/node/1242856 Core Office Hours]. If this sounds like something you'd like to contribute to, don't hesitate to add your name!

{| border="1" cellspacing="0" cellpadding="5" align="center"
! Mentor name
! IRC name
! Social
! Willing to help with
! Looking to learn
! Office hours
|-
| [[User:Mdm|Matt McCollow]]
| mdm_
|
| Javascript
|
| Fridays, 1-2 pm EST
|-
| [http://ruebot.net Nick Ruest]
| ruebot
| [http://twitter.com/ruebot Twitter] [https://plus.google.com/107824412266039153440 G+]
| Islandora
| Getting better with Solr
| Fridays, 1-2 pm EST (pls ping in advance)
|-
| [http://jbfink.github.com John Fink]
| jbfink
|
| git
|
| Fridays, 1-2pm EST
|-
| [http://lackoftalent.org/michael/ Mike Giarlo]
| mjgiarlo
| [http://twitter.com/mjgiarlo t] [http://gplus.to/mjgiarlo G+] [http://facebook.com/mjgiarlo fb]
| Repository technology and architecture, Ruby, Git, Project Hydra, "Persistent identifiers"
|
| Fridays, 1-5 pm EST (pls ping in advance)
|-
| [http://kennison.name/ Brian Kennison]
| briankenn
|
| xslt
|
| Fridays, 1-2pm EST (let me know what works for you)
|-
| [http://github.com/tdonohue/ Tim Donohue]
| tdonohue
| [http://twitter.com/timdonohue Twitter]
| DSpace, repository technologies/integrations in general, SWORD protocol, digital preservation stuff
|
| [http://wiki.duraspace.org/display/~tdonohue/DSpace+Office+Hours Weds, 1-4pm ET in #duraspace channel on IRC Freenode]
|-
|}

Mentorship

2012-06-04T17:49:28Z

Tdonohue:

We're looking to try out a mentorship program through the #code4lib channel on IRC Freenode. The idea is that experienced people who are willing to volunteer some time would provide 1-on-1 mentoring to those who would like to learn during an "Office hours" period, once a week. The idea is sort-of modelled on the Drupal project's [http://drupal.org/node/1242856 Core Office Hours]. If this sounds like something you'd like to contribute to, don't hesitate to add your name!

{| border="1" cellspacing="0" cellpadding="5" align="center"
! Mentor name
! IRC name
! Social
! Willing to help with
! Office hours
|-
| [[User:Mdm|Matt McCollow]]
| mdm_
|
| Javascript
| Fridays, 1-2 pm EST
|-
| [http://ruebot.net Nick Ruest]
| ruebot
| [http://twitter.com/ruebot Twitter] [https://plus.google.com/107824412266039153440 G+]
| Islandora
| Fridays, 1-2 pm EST (pls ping in advance)
|-
| [http://jbfink.github.com John Fink]
| jbfink
|
| git
| Fridays, 1-2pm EST
|-
| [http://lackoftalent.org/michael/ Mike Giarlo]
| mjgiarlo
|
| Repository technology and architecture, Ruby, Git, Project Hydra, "Persistent identifiers"
| Fridays, 1-2 pm EST (though pls ping in advance)
|-
| [http://kennison.name/ Brian Kennison]
| briankenn
|
| xslt
| Fridays, 1-2pm EST (let me know what works for you)
|-
| [http://github.com/tdonohue/ Tim Donohue]
| tdonohue
| [http://twitter.com/timdonohue Twitter]
| DSpace, repository technologies/integrations in general
| [http://wiki.duraspace.org/display/~tdonohue/DSpace+Office+Hours Weds, 1-4pm ET in #duraspace channel on IRC Freenode]
|-
|}

HAMR: Human/Authority Metadata Reconciliation

2011-02-07T21:13:12Z

Tdonohue:

[[HAMR: Human/Authority Metadata Reconciliation]]

Sean Chen, Tim Donohue, Joshua Gomez, Ranti Junus, Ryan Scherle

A tool for a curator to determine whether the various fields of a metadata record are correct. Takes a metadata record, locates any identifiers (e.g., DOI, PMID). Retrieves a copy of the metadata record from an authoritative source (e.g., CrossRef, PubMed). Displays a human-readable page that compares fields in the initial record with fields in the authoritative record. Each field is color-coded based on how well it matches, so the curator can quickly identify discrepancies.

Narrowing the focus for today:
* Dublin core (maybe qualified)
* framework that allows multiple authority sources
* NOT focusing on author names ([http://www.orcid.org/ ORCID] is already working on this), except the fact that they are strings, and we'll do basic string matching
* 1 to 1 matching. Even if you want to eventually match with multiple authorities, you'd only do one at a time

Possible authority sources:
* PubMed
** Sample pubmed query (in Java): [https://wiki.duraspace.org/display/DSPACE/PubMedPrefill-PubmedPrefillStep.java DSpace PubMedPrefillStep.java] (From [https://wiki.duraspace.org/display/DSPACE/PopulateMetadataFromPubMed Populate Metadata from PubMed])
*** See 'retrievePubmedXML()' in above java code for actual call to PubMed
*** Mapping happens here: See [https://wiki.duraspace.org/display/DSPACE/PubMedPrefill-pmid+dim.xsl pmid-to-dim.xsl] for a sample XSLT crosswalk to translate PubMed format to a qualified dublin core (internal DSpace metadata format)
** More examples of querying PubMed: http://www.my-whiteboard.com/how-to-automate-pubmed-search-using-perl-php-or-java/
* CrossRef
** simply send the DOI to crossref, and get JSON/XML back
*** http://api.labs.crossref.org/10.1111/j.1558-5646.2009.00626.x.json
*** http://api.labs.crossref.org/10.2307/1935157.xml
*** [http://code.google.com/p/dryad/source/browse/trunk/dryad/dspace/modules/doi/dspace-doi-webapp/src/main/java/org/dspace/doi/DOIServlet.java java code that includes a lookup]
** [http://labs.crossref.org/site/crossref_metadata_search.html Metadata Search] -- send a text query, receive a list of matching records
** [http://labs.crossref.org/site/quick_and_dirty_api_guide.html OpenURL search]
* google scholar - does it have an API?
* [http://www.mendeley.com mendeley] - [http://dev.mendeley.com/ Mendeley API]
* [http://vivoweb.org/ vivo]
* [http://bibapp.org/ bibapp]

Thoughts / Questions:
* Is there a way to do most/all of this via Javascript/AJAX/JQuery? Could it be a simple Javascript framework you could "drop" into any metadata editing interface?
** Unfortunately, it seems this wouldn't work out. In order to perform querying of external authorities, they'd all need to support [http://en.wikipedia.org/wiki/JSON#JSONP JSONP] or similar (and they don't)

== Code ==

* [http://gitref.org/ quick reference for Git]
* [https://github.com/ryscher/hamr Ryan's really stupid scratch implementation]

== Output Spec ==

* We will use a simple XML output consisting of paired (and possibly unpaired) values.
* The root element will contain an attribute signifying the source of the authority metadata.
* The <match> element will be used to pair values, with a strength attribute to signify the string distance.
* Within each match element will be exactly 2 metadata elements with attributes signifying the source of each value: either the local input or the remote authority data.
* An <nonmatch> element will be used for unpaired values.

=== Sample Output ===
<pre>
<hamr authority="PubMed">
<match strength="100%">
<creator src="input">Trojan, Tommy</creator>
<creator src="authority">Trojan, Tommy</creator>
</match>
<match strength="90%">
<title src="input">Great American Article</title>
<title src="authority">Great American Article, The</title>
</match>
<nonmatch>
<subject src="input">Medical Stuff</subject>
</nonmatch>
<nonmatch>
<type src="authority">text</type>
</nonmatch>
</hamr>
</pre>

== Need to do ==

# Implement metadata retrieval from authority ''(done for crossref in ryan's code)''
# Design structure of plugins
## crosswalk from authority format to simple dc
# Design matching algorithm

HAMR: Human/Authority Metadata Reconciliation

2011-02-07T19:13:09Z

Tdonohue:

HAMR: Human/Authority Metadata Reconciliation

2011-02-07T16:04:43Z

Tdonohue:

HAMR: Human/Authority Metadata Reconciliation

2011-02-07T15:58:34Z

Tdonohue:

HAMR: Human/Authority Metadata Reconciliation

2011-02-07T15:49:58Z

Tdonohue:

HAMR: Human/Authority Metadata Reconciliation

2011-02-07T15:45:43Z

Tdonohue:

HAMR: Human/Authority Metadata Reconciliation

2011-02-07T15:43:08Z

Tdonohue:

HAMR: Human/Authority Metadata Reconciliation

2011-02-07T15:33:00Z

Tdonohue:

HAMR: Human/Authority Metadata Reconciliation

2011-02-07T15:32:47Z

Tdonohue:

HAMR: Human/Authority Metadata Reconciliation

2011-02-07T15:32:26Z

Tdonohue: