Difference between revisions of "Hacking Pre-Ingest Assessment Tools (Solr/Ruby/Python)"
| (9 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
'''Django/Solr Metadata Archive Tool''' | '''Django/Solr Metadata Archive Tool''' | ||
| − | As part of my [http://code4lib.org/conference/2011/Matienzo code4lib presentation] I may demo some code that works with Digital Forensics XML and gets it into a Solr index. I've successfully thrown Blacklight on top of it, but want to extend it further, especially in terms of figuring what I can do with it and creating a straightforward UI that will represent directory hierarchies. | + | As part of my [http://code4lib.org/conference/2011/Matienzo code4lib presentation] I (Matienzo) may demo some code that works with Digital Forensics XML and gets it into a Solr index. I've successfully thrown Blacklight on top of it, but want to extend it further, especially in terms of figuring what I can do with it and creating a straightforward UI that will represent directory hierarchies. |
* https://github.com/anarchivist/foresole | * https://github.com/anarchivist/foresole | ||
* https://github.com/anarchivist/gumshoe | * https://github.com/anarchivist/gumshoe | ||
| + | * Solr index w/ sample data: http://solr.onebigarchives.net:8983/solr/admin/ | ||
| + | * Sample query: http://solr.onebigarchives.net:8983/solr/select?indent=on&version=2.2&q=*:*&fq=&start=0&rows=10&fl=*,score&qt=standard&wt=standard&explainOther=&hl.fl= | ||
| + | |||
| + | This would maybe be happy with an Event microservice. Mark Phillips hopes to release a Django app to this effect in April 2011. | ||
| + | |||
| + | ==Fears== | ||
| + | * Identifiers are precious | ||
| + | * Ingest is forever | ||
| + | * Where does rights management come in | ||
| + | * Hard drives full of junk and an uncorrelated spreadsheet. | ||
| + | * Resolving logical conflicts in human-edited spreadsheets--often difficult to notice problems in advance | ||
| + | |||
| + | ==Desires== | ||
| + | * Command-line statistical analysis (histogram, number of distinct values, etc.) of spreadsheets. | ||
| + | * Organizable digital limbo | ||
| + | * Pie charts and other visualizations. (How much of this stuff is ingestable? and other questions) | ||
| + | |||
| + | ==Tools== | ||
| + | * Event microservice | ||
| + | * GUI XSLT editors exist for MARC... how about for spreadsheets? | ||
Latest revision as of 20:12, 7 February 2011
Django/Solr Metadata Archive Tool
As part of my code4lib presentation I (Matienzo) may demo some code that works with Digital Forensics XML and gets it into a Solr index. I've successfully thrown Blacklight on top of it, but want to extend it further, especially in terms of figuring what I can do with it and creating a straightforward UI that will represent directory hierarchies.
- https://github.com/anarchivist/foresole
- https://github.com/anarchivist/gumshoe
- Solr index w/ sample data: http://solr.onebigarchives.net:8983/solr/admin/
- Sample query: http://solr.onebigarchives.net:8983/solr/select?indent=on&version=2.2&q=*:*&fq=&start=0&rows=10&fl=*,score&qt=standard&wt=standard&explainOther=&hl.fl=
This would maybe be happy with an Event microservice. Mark Phillips hopes to release a Django app to this effect in April 2011.
Fears
- Identifiers are precious
- Ingest is forever
- Where does rights management come in
- Hard drives full of junk and an uncorrelated spreadsheet.
- Resolving logical conflicts in human-edited spreadsheets--often difficult to notice problems in advance
Desires
- Command-line statistical analysis (histogram, number of distinct values, etc.) of spreadsheets.
- Organizable digital limbo
- Pie charts and other visualizations. (How much of this stuff is ingestable? and other questions)
Tools
- Event microservice
- GUI XSLT editors exist for MARC... how about for spreadsheets?