3-5-Database.tex

\subsection{The File Database}\label{ssec:db}

Easy access to large quantities of solar data frequently leads to data files accumulating
in local storage such as laptops and desktop computers. Keeping data organised and available
is typically a cumbersome task for the average user. The file database is a subpackage of 
SunPy that addresses this problem by providing a unified database to store and 
manage information about local data files.

The \texttt{database} subpackage can make use of any database software supported by
\texttt{SQLAlchemy} (\url{http://www.sqlalchemy.org}). This library was chosen
since it supports many SQL dialects. 
If SQLite is selected, the database is stored as a single file, which is
created automatically. A server-based database, on the other hand, could be used
by collaborators who work together on the
same data from different computers: a central database server stores all data and the clients connect to
it to read or write data.

The database can store and manage all data that can be read via SunPy's 
\texttt{io} subpackage, and direct integration with the \texttt{vso} 
module is supported.
It is also possible to manually add file or directory entries. The package also provides
a unified data search via the \texttt{fetch()} method, which includes both local files
and files on the \textsc{VSO}. This reduces the likelihood of downloading the same file 
multiple times. When a file is added to the database, the file is scanned for metadata,
and a file hash is produced. 
The current date is associated with the entry along with metadata summaries such 
as instrument, date of observation, field of view, etc. 
The database also provides the ability to associate custom metadata to 
each database entry such as keywords, comments, and favourite tags, as well as 
%dps: favouring is not what we want to say here...
%JI: ``assigning favourites'' - is this what is meant?
%dps: Yes, how does it sounds now?
querying the full metadata (e.g., FITS header) of each entry.

The \texttt{Database} class connects to a database and allows the user to 
perform operations on it. Listing~\ref{code:database} shows how to connect
to an in-memory database and download data from the \textsc{VSO}. These entries are
automatically added to the database. The function \texttt{len()} is used to get the number of
records. The function \texttt{display\_entries()} displays an iterable of 
database entries in a formatted \textsc{ASCII} table. The headlines 
correspond to the attributes of the respective database entries.

A useful feature of the database package is the support of \texttt{undo}
and \texttt{redo} operations. This is particularly convenient in
interactive sessions to easily revert accidental operations. 
This feature will also be desirable for a planned GUI frontend for this package.

\begin{listing}[H]
\pythoncode{pycode_database.txt}
\caption{Example usage of the \texttt{database} subpackage.}
\label{code:database}
\end{listing}