◊(Local Yarn Code "cache.scrbl at [34d3bec4]")

File code-docs/cache.scrbl artifact b6327b43 part of check-in 34d3bec4


#lang scribble/manual

@; SPDX-License-Identifier: BlueOak-1.0.0
@; This file is licensed under the Blue Oak Model License 1.0.0.

@(require "scribble-helpers.rkt" scribble/example)

@(require (for-label deta
                     db
                     racket/base
                     racket/contract
                     sugar/coerce
                     pollen/template
                     "../dust.rkt"
                     "../crystalize.rkt"
                     "../cache.rkt"))

@(define example-eval (make-base-eval))
@(example-eval '(require "cache.rkt" txexpr))

@title[#:tag "cache-rkt"]{Cache}

@defmodule["cache.rkt" #:packages ()]

In this project there are several places – the blog, the footer on each page, the RSS feed, series
pages — where data from an amorphous group of Pollen documents is needed. This is what the cache is
for.

This module defines and provides the schema and database connection to the SQLite cache, and some
functions for retrieving records from the cache. Use these when you need quick access to pre-cooked
HTML.

@section{Cache database}

@defparam[cache-conn conn connection?]{
The database connection.
}

@defproc[(init-cache-db!) void?]{

Creates and initializes the SQLite database cache file (named @filepath{vitreous.sqlite} and located
in the project root folder) by running queries to create tables in the database if they do not
exist.

}

@section{Retrieving cached data}

Some of this looks a little wacky, but it’s a case of putting a little extra complextity into the
back end to make things simple on the front end. These functions are most commonly used inside the
@emph{body} of a Pollen document (i.e., series pages). 

@filebox["series/my-series.poly.pm"
@codeblock|{
#lang pollen

◊title{My New Series}

...some other content

◊fenced-listing[(articles+notes 'excerpt #:order 'asc)]
}|
]

@defproc[(fenced-listing [query query?]) txexpr?]{

Fetches a the HTML strings from the SQLite cache and returns a @racket['style] tagged X-expression
with these strings as its elements. The @racket[_query] will usually be the result of a call to
@racket[articles] or @racket[articles+notes], but can be any custom query that projects onto the
@racket[listing] schema (see @racket[project-onto]).

The reason for enclosing the results in a @racket['style] txexpr is to prevent the HTML from being
escaped by @racket[->html] in the template. This tag was picked for the job because there will
generally never be a need to include any actual CSS information inside a @tt{<style>} tag in any
page, so it can be safely filtered out later. To remove the enclosing @tt{<style>} tag, see
@racket[unfence].

}

@defproc[(listing-htmls [listing-query query?]) (listof string?)]{

Returns the HTML bodies for the articles and/or notes returned by @racket[_listing-query] as a list
of strings. The @racket[_query] will usually be the result of a call to @racket[articles] or
@racket[articles+notes], but can be any custom query that projects onto the @racket[listing] schema
(see @racket[project-onto]).

}

@deftogether[(@defproc[(articles [type (or/c 'full 'excerpt 'short 'content)]
                                 [#:series series (or/c string? (listof string?) boolean?) #t]
                                 [#:limit limit integer? -1]
                                 [order stringish? 'desc]) query?]
              @defproc[(articles+notes [type (or/c 'full 'excerpt 'short 'content)]
                                       [#:series series (or/c string? (listof string?) boolean?) #t]
                                       [#:limit limit integer? -1]
                                       [order stringish? 'desc]) query?])]{

Create a query that fetches either articles only, or articles and notes intermingled, respectively.
The results will be sorted by publish date according to @racket[_order] and optionally limited to
a particular series. Use the resulting query with the @racket[listing-htmls] or
@racket[fenced-listing] functions provided by this module, or with deta’s @racket[in-entities] if
you want to work with the @racket[listing] schema structs.

The @racket[_type] parameter specifies what version of the articles’ and notes’ HTML markup you
want. For HTML suitable for listing several articles and/or notes together on the same page, use
@racket['full] (the full content but not including @tech{notes}), @racket['excerpt] (like full but
abbreviated to only the excerpt if one was specified) or @racket['short] (date and title only). Use
@racket['content] to get the entire HTML content, including any notes but not including any header
or footer. (This is the option used in the RSS feed.)

If @racket[_series] expression evaluates to @racket[#f], articles will not be filtered by series. If
it evaluates to @racket[#t] (the default), articles will be filtered by those that specify the
current output of @racket[here-output-path] in their @tt{series_pagenode} column in the SQLite
cache. If a string or a symbol is supplied, articles will be filtered by those containing the result
of @racket[(format "series/~a.html" _series)] in their @tt{series_pagenode} column in the SQLite
cache. If a list of strings or symbols is provided, this @racket[format] operation will be applied
to each of its members and articles whose @tt{series_pagenode} column matches any of the resulting
values will be included.

The @racket[_order] expression must evaluate to either @racket["ASC"] or @racket["DESC"] (or
equivalent symbols) and the @racket[_limit] expressions must evaluate to a value suitable for use in
the @tt{LIMIT} clause of @ext-link["https://sqlite.org/lang_select.html"]{a SQLite @tt{SELECT}
statement}. An expression that evaluates to a negative integer (the default) is the same as having
no limit.

Typically you will pass these functions by name to listing functions like @racket[fenced-listing]
rather than calling them directly.

@examples[#:eval example-eval
(articles 'full)]
}

@defproc[(unfence [html string?]) string?]{

Returns @racket[_html] with all occurrences of @racket["<style>"] and @racket["</style>"] removed.
The contents of the style tags are left intact.

Use this in templates with strings returned from @racket[->html] when called on docs that use the
@racket[fenced-listing] tag function.

}

@section{Modifying the cache}

@defproc[(save-cache-things! 
          [things (listof (or/c cache:article? cache:note? cache:index-entry?))]) void?]{

Saves all the @racket[_thing]s to the cache database.

}

@deftogether[(@defproc[(delete-article! [page stringish?]) void?]
              @defproc[(delete-notes!   [page stringish?]) void?])]{
              
Delete a particular article, or all notes for a particular article, respectively.

}

@section{Schema}

The cache database has four tables: @tt{articles}, @tt{notes}, @tt{index_entries} and @tt{series}.
Each of these has a corresponding schema, shown below. In addition, there is a “virtual” schema,
@tt{listing}, for use with queries which may or may not combine articles and notes intermingled.

The work of picking apart an article’s exported @tt{doc} and @tt{metas} into rows in these tables is
done by @racket[parse-and-cache-article!].

The below are shown as @code{struct} forms but are actually defined with deta’s
@racket[define-schema]. Each schema has an associated struct with the same name and a smart
constructor called @tt{make-@emph{id}}. The struct’s “dumb” constructor is hidden so that invalid
entities cannot be created. For every defined field there is an associated functional setter and
updater named @tt{set-@emph{id}-field} and @tt{update-@emph{id}-field}, respectively.

@defstruct*[cache:article ([id                   id/f]
                           [page                 symbol/f]
                           [title-plain          string/f]
                           [title-html-flow      string/f]
                           [title-specified      boolean/f]
                           [published            string/f]
                           [updated              string/f]
                           [author               string/f]
                           [conceal              string/f]
                           [series-page          string/f]
                           [noun-singular        string/f]
                           [note-count           integer/f]
                           [content-html         string/f]
                           [disposition          string/f]
                           [disp-html-anchor     string/f]
                           [listing-full-html    string/f]
                           [listing-excerpt-html string/f]
                           [listing-short-html   string/f])
            #:constructor-name make-cache:article]{

Table holding cached article information.

}

@defstruct*[cache:note ([id              id/f]
                        [page            symbol/f]
                        [html-anchor     string/f]
                        [title-html-flow string/f]
                        [title-plain     string/f]
                        [author          string/f]
                        [author-url      string/f]
                        [published       string/f]
                        [disposition     string/f]
                        [content-html    string/f]
                        [series-page     symbol/f]
                        [conceal         string/f]
                        [listing-full-html    string/f]
                        [listing-excerpt-html string/f]
                        [listing-short-html   string/f])
                       #:constructor-name make-cache:note]{

Table holding cached information on notes.

}

@defstruct*[cache:index-entry ([id id/f]
                               [entry       string/f]
                               [subentry    string/f]
                               [page        symbol/f]
                               [html-anchor string/f])
                              #:constructor-name make-cache:index-entry]{
                              
Table holding cached information about index entries found in articles.

}

@defstruct*[listing ([path        string/f]
                     [title       string/f]
                     [author      string/f]
                     [published   string/f]
                     [updated     string/f]
                     [html        string/f])
                    #:constructor-name make-listing]{

This is a “virtual” schema targeted by @racket[articles] and @racket[articles+notes] using deta’s
@racket[project-onto]. It supplies the minimum set of fields needed to build the RSS feed, and which
are common to both articles and notes; most times (e.g., on @tech{series} pages) only the @tt{html}
field is used, via @racket[fenced-listing] or @racket[listing-htmls].

}