ADDED   code-docs/cache.scrbl
Index: code-docs/cache.scrbl
==================================================================
--- code-docs/cache.scrbl
+++ code-docs/cache.scrbl
@@ -0,0 +1,223 @@
+#lang scribble/manual
+
+@; SPDX-License-Identifier: BlueOak-1.0.0
+@; This file is licensed under the Blue Oak Model License 1.0.0.
+
+@(require "scribble-helpers.rkt" scribble/example)
+
+@(require (for-label deta
+                     db
+                     racket/base
+                     racket/contract
+                     sugar/coerce
+                     pollen/template
+                     "../dust.rkt"
+                     "../crystalize.rkt"
+                     "../cache.rkt"))
+
+@(define example-eval (make-base-eval))
+@(example-eval '(require "cache.rkt" txexpr))
+
+@title[#:tag "cache-rkt"]{Cache}
+
+@defmodule["cache.rkt" #:packages ()]
+
+In this project there are several places – the blog, the footer on each page, the RSS feed, series
+pages — where data from an amorphous group of Pollen documents is needed. This is what the cache is
+for.
+
+This module defines and provides the schema and database connection to the SQLite cache, and some
+functions for retrieving records from the cache. Use these when you need quick access to pre-cooked
+HTML.
+
+@section{Cache database}
+
+@defthing[cache-conn connection?]{
+The database connection.
+}
+
+@defproc[(init-cache-db!) void?]{
+Creates and initializes the SQLite database cache file (named @filepath{vitreous.sqlite} and located
+in the project root folder) by running queries to create tables in the database if they do not
+exist.
+
+This function is called automatically in @seclink["pollen-rkt"]{@filepath{pollen.rkt}} whenever HTML
+is the target output. }
+
+@defparam[current-plain-title title-plain non-empty-string? #:value "void"]{
+Contains (or sets) the “plain” title (i.e., with no HTML markup) for the current article based on
+analysis done by @racket[parse-and-cache-article!]. If the article did not specify a title,
+a default title is supplied. If the article contained a @racket[note] that used the
+@code{#:disposition} attribute, the disposition-mark may be included in the title.
+
+This is a weird parameter, and at some point I will probably get rid of it and have
+@racket[parse-and-cache-article!] supply it as an extra return value instead.
+
+@margin-note{Note that this needs to be called @emph{after} @racket[parse-and-cache-article!] in
+order to get an up-to-date value.}
+}
+
+@section{Retrieving cached data}
+
+Some of this looks a little wacky, but it’s a case of putting a little extra complextity into the
+back end to make things simple on the front end. These functions are most commonly used inside the
+@emph{body} of a Pollen document (i.e., series pages). 
+
+@filebox["series/my-series.poly.pm"
+@codeblock|{
+#lang pollen
+
+◊title{My New Series}
+
+...some other content
+
+◊(<listing-excerpt> articles+notes #:order 'asc)
+}|
+]
+
+@deftogether[(@defproc[(<listing-full> 
+                                 [query-func (-> any/c query?)]
+                                 [#:series series (or/c string? (listof string?) boolean?) #t]
+                                 [#:limit limit integer? -1]
+                                 [order stringish? 'desc]) txexpr?]
+              @defproc[(<listing-excerpt>
+                                 [query-func (-> any/c query?)]
+                                 [#:series series (or/c string? (listof string?) boolean?) #t]
+                                 [#:limit limit integer? -1]
+                                 [order stringish? 'desc]) txexpr?]
+              @defproc[(<listing-short>
+                                 [query-func (-> any/c query?)]
+                                 [#:series series (or/c string? (listof string?) boolean?) #t]
+                                 [#:limit limit integer? -1]
+                                 [order stringish? 'desc]) txexpr?])]{
+Fetches the HTML for items from the SQLite cache, concatenates their HTML strings and returns
+a @racket['style] tagged X-expression with this string as its element. The items will be ordered by
+publish date according to @racket[_order] and optionally limited to the series specified in
+@racket[_series].
+
+The @racket[_query-func] should be either @racket[articles], which will create a listing of articles
+only, or @racket[articles+notes], which will include notes intermingled with articles.
+
+@margin-note{Note that the signature shown for the @racket[_query-func] argument above is
+incomplete. If you choose to pass a function other than @racket[articles] or
+@racket[articles+notes], you must use a function with exactly the same signature as those
+functions.}
+
+If @racket[_series] expression evaluates to @racket[#f], articles will not be filtered by series. If
+it evaluates to @racket[#t] (the default), articles will be filtered by those that specify the
+current output of @racket[here-output-path] in their @tt{series_pagenode} column in the SQLite
+cache. If a string is supplied, articles will be filtered by those containing that exact value in
+their @tt{series_pagenode} column in the SQLite cache.
+
+The @racket[_order] expression must evaluate to either @racket["ASC"] or @racket["DESC"] and the
+@racket[_limit] expressions must evaluate to a value suitable for use in the @tt{LIMIT} clause of
+@ext-link["https://sqlite.org/lang_select.html"]{a SQLite @tt{SELECT} statement}. An expression that
+evaluates to a negative integer (the default) is the same as having no limit.
+
+The reason for enclosing the results in a @racket['style] txexpr is to prevent the HTML from being
+escaped by @racket[->html] in the template. This tag was picked for the job because there will
+generally never be a need to include any actual CSS information inside a @tt{<style>} tag in any
+page, so it can be safely filtered out later. To remove the enclosing @tt{<style>} tag, see
+@racket[unfence].
+}
+
+@defproc[(listing-htmls [listing-query query?]) (listof string?)]{
+Returns the HTML bodies for the articles and/or notes returned by @racket[_listing-query] as a list
+of strings.
+}
+
+@deftogether[(@defproc[(articles [type (or/c 'full 'excerpt 'short)]
+                                 [#:series series (or/c string? (listof string?) boolean?) #t]
+                                 [#:limit limit integer? -1]
+                                 [order stringish? 'desc]) query?]
+              @defproc[(articles+notes [type (or/c 'full 'excerpt 'short)]
+                                       [#:series series (or/c string? (listof string?) boolean?) #t]
+                                       [#:limit limit integer? -1]
+                                       [order stringish? 'desc]) query?])]{
+Create a query that fetches either articles only (@racket[articles]) or articles and notes
+intermingled (@racket[articles+notes]), sorted by publish date and optionally limited to
+a particular series.
+
+Typically you will pass these functions by name to listing functions like @racket[<listing-full>]
+rather than calling them directly.
+
+@examples[#:eval example-eval
+(articles 'full)]
+}
+
+@defproc[(unfence [html string?]) string?]{
+Returns @racket[_html] with all occurrences of @racket["<style>"] and @racket["</style>"] removed.
+The contents of the style tags are left intact.
+
+Use this in templates with strings returned from @racket[->html] when called on docs that use the
+@racket[<listing-full>] tag function or its siblings. }
+
+@defproc[(series-grouped-list) (listof (listof cache:series?))]{
+Return a list of lists of all @racket[cache:series] in the cache database. The series are grouped so
+that series using the same value in the @tt{noun-plural} column appear together.}
+
+@section{Deleting records}
+
+@deftogether[(@defproc[(delete-article! [page stringish?]) void?]
+              @defproc[(delete-notes!   [page stringish?]) void?])]{
+Delete a particular article, or all notes for a particular article, respectively.
+}
+
+@section{Schema}
+
+The cache database has four tables: @tt{articles}, @tt{notes}, @tt{index_entries} and @tt{series}. Each of these has a corresponding schema, shown below. In addition, there is a “virtual” schema, @tt{listing}, for use with queries which may or may not combine articles and notes intermingled.
+
+The work of picking apart an article’s exported @tt{doc} and @tt{metas} into rows in these tables is done by @racket[parse-and-cache-article!].
+
+The below are shown as @code{struct} forms but are actually defined with deta’s @racket[define-schema]. Each schema has an associated struct with the same name and a smart constructor called @tt{make-@emph{id}}. The struct’s “dumb” constructor is hidden so that invalid entities cannot be created. For every defined field there is an associated functional setter and updater named @tt{set-@emph{id}-field} and @tt{update-@emph{id}-field}, respectively.
+
+@defstruct*[cache:article ([id                   id/f]
+                           [page                 symbol/f]
+                           [title-plain          string/f]
+                           [title-html-flow      string/f]
+                           [title-specified      boolean/f]
+                           [published            string/f]
+                           [updated              string/f]
+                           [author               string/f]
+                           [conceal              string/f]
+                           [series-page          string/f]
+                           [noun-singular        string/f]
+                           [note-count           integer/f]
+                           [doc-html             string/f]
+                           [disposition          string/f]
+                           [disp-html-anchor     string/f]
+                           [listing-full-html    string/f]
+                           [listing-excerpt-html string/f]
+                           [listing-short-html   string/f])
+            #:constructor-name make-cache:article]{
+Table holding cached article information.
+}
+
+@defstruct*[cache:note ([id              id/f]
+                        [page            symbol/f]
+                        [html-anchor     string/f]
+                        [title-html-flow string/f]
+                        [title-plain     string/f]
+                        [author          string/f]
+                        [author-url      string/f]
+                        [published       string/f]
+                        [disposition     string/f]
+                        [content-html    string/f]
+                        [series-page     symbol/f]
+                        [conceal         string/f]
+                        [listing-full-html    string/f]
+                        [listing-excerpt-html string/f]
+                        [listing-short-html   string/f])
+                       #:constructor-name make-cache:note]{
+Table holding cached information on notes.
+}
+
+@defstruct*[cache:series ([id            id/f]
+                          [page          symbol/f]
+                          [title         string/f]
+                          [published     string/f]
+                          [noun-plural   string/f]
+                          [noun-singular string/f])
+                         #:constructor-name make-cache:series]{
+Table holding cached-information on series.
+}

Index: code-docs/crystalize.scrbl
==================================================================
--- code-docs/crystalize.scrbl
+++ code-docs/crystalize.scrbl
@@ -9,15 +9,12 @@
                      "../dust.rkt"
                      "../crystalize.rkt"
                      racket/base
                      racket/contract
                      racket/string
-                     deta
                      txexpr
-                     pollen/template
-                     pollen/pagetree
-                     sugar/coerce))
+                     pollen/pagetree))
 
 @title{@filepath{crystalize.rkt}}
 
 @defmodule["crystalize.rkt" #:packages ()]
 
@@ -25,18 +22,10 @@
 it in various pieces in a SQLite cache. Individual articles save chunks of rendered HTML to the
 cache when their individual pages are rendered. When pulling together listings of articles in
 different contexts that need to be filtered and sorted, a SQL query is much faster than trolling
 through the Pollen cache for matching docs and regenerating the HTML.
 
-@defproc[(init-cache-db!) void?]
-
-Initializes the SQLite database cache file (named @filepath{vitreous.sqlite} and located in the
-project root folder) by running queries to create tables in the database if they do not exist. (The
-file itself is created at the module level.)
-
-This function is called automatically in @filepath{pollen.rkt} whenever HTML is the target output.
-
 @defproc[(parse-and-cache-article! [pagenode pagenode?] [doc txexpr?]) non-empty-string?]
 
 Returns a string containing the HTML of @racket[_doc]. @margin-note{This is one function that breaks
 my convention of using a prefix of @tt{html$-} for functions that return a single string of HTML.}
 
@@ -45,83 +34,5 @@
 about that series is fetched and used in the rendering of the article. If there are @racket[note]s
 in the doc, they are parsed and saved individually to the SQLite cache. If any of the notes use the
 @code{#:disposition} attribute, information about the disposition is parsed out and used in the
 rendering of the article.
 
-@deftogether[(@defproc[(<listing-full> 
-                                 [query-func (-> any/c query?)]
-                                 [#:series series (or/c string? (listof string? boolean?)) #t]
-                                 [#:limit limit integer? -1]
-                                 [order stringish? 'desc]) txexpr?]
-              @defproc[(<listing-excerpt>
-                                 [query-func (-> any/c query?)]
-                                 [#:series series (or/c string? (listof string? boolean?)) #t]
-                                 [#:limit limit integer? -1]
-                                 [order stringish? 'desc]) txexpr?]
-              @defproc[(<listing-short>
-                                 [query-func (-> any/c query?)]
-                                 [#:series series (or/c string? (listof string? boolean?)) #t]
-                                 [#:limit limit integer? -1]
-                                 [order stringish? 'desc]) txexpr?])]
-
-Fetches the HTML for items from the SQLite cache and returns the HTML strings fenced inside
-a @racket['style] tagged X-expression. The items will be ordered by publish date according to
-@racket[_order] and optionally limited to the series specified in @racket[_series].
-
-The @racket[_query-func] should be either @racket[articles], which will create a listing of articles
-only, or @racket[articles+notes], which will include notes intermingled with articles.
-
-@margin-note{Note that the signature shown for the @racket[_query-func] argument above is
-incomplete. If you choose to pass a function other than @racket[articles] or
-@racket[articles+notes], you must use a function with exactly the same signature as those
-functions.}
-
-If @racket[_series] expression evaluates to @racket[#f], articles will not be filtered by series. If
-it evaluates to @racket[#t] (the default), articles will be filtered by those that specify the
-current output of @racket[here-output-path] in their @tt{series_pagenode} column in the SQLite
-cache. If a string is supplied, articles will be filtered by those containing that exact value in
-their @tt{series_pagenode} column in the SQLite cache.
-
-The @racket[_order] expression must evaluate to either @racket["ASC"] or @racket["DESC"] and the
-@racket[_limit] expressions must evaluate to a value suitable for use in the @tt{LIMIT} clause of
-@ext-link["https://sqlite.org/lang_select.html"]{a SQLite @tt{SELECT} statement}. An expression that
-evaluates to a negative integer (the default) is the same as having no limit.
-
-The reason for enclosing the results in a @racket['style] txexpr is to prevent the HTML from being
-escaped by @racket[->html] in the template. This tag was picked for the job because there will
-generally never be a need to include any actual CSS information inside a @tt{<style>} tag in any
-page, so it can be safely filtered out later. To remove the enclosing @tt{<style>} tag, see
-@racket[unfence].
-
-@deftogether[(@defproc[(articles [type (or/c 'full 'excerpt 'short)]
-                                 [#:series series (or/c string? (listof string? boolean?)) #t]
-                                 [#:limit limit integer? -1]
-                                 [order stringish? 'desc]) query?]
-              @defproc[(articles+notes [type (or/c 'full 'excerpt 'short)]
-                                       [#:series series (or/c string? (listof string? boolean?)) #t]
-                                       [#:limit limit integer? -1]
-                                       [order stringish? 'desc]) query?])]
-
-Create a query that fetches either articles only (@racket[articles]) or articles and notes
-intermingled (@racket[articles+notes]), sorted by publish date and optionally limited to
-a particular series.
-
-Typically you will pass these functions by name to listing functions like @racket[<listing-full>]
-rather than calling them directly.
-
-@defproc[(unfence [html string?]) string?]
-
-Returns @racket[_html] with all occurrences of @racket["<style>"] and @racket["</style>"] removed.
-The contents of the style tags are left intact.
-
-Use this with strings returned from @racket[->html] when called on docs that use the
-@racket[<listing-full>] tag function or its siblings.
-
-@defparam[current-plain-title non-empty-string? #:value "void"]
-
-Contains (or sets) the “plain” title (i.e., with no HTML markup) for the current article based on
-analysis done by @racket[parse-and-cache-article!]. If the article did not specify a title,
-a default title is supplied. If the article contained a @racket[note] that used the
-@code{#:disposition} attribute, the disposition-mark may be included in the title.
-
-Note that this needs to be called @emph{after} @racket[parse-and-cache-article!] in order to get an
-up-to-date value.

Index: code-docs/dust.scrbl
==================================================================
--- code-docs/dust.scrbl
+++ code-docs/dust.scrbl
@@ -6,10 +6,11 @@
 @(require "scribble-helpers.rkt"
           scribble/example)
 
 @(require (for-label "../pollen.rkt"
                      "../dust.rkt"
+                     "../cache.rkt"
                      racket/base
                      racket/contract
                      txexpr
                      sugar/coerce
                      pollen/tag
@@ -18,11 +19,11 @@
                      pollen/core))
 
 @(define dust-eval (make-base-eval))
 @(dust-eval '(require "dust.rkt" txexpr))
 
-@title{@filepath{dust.rkt}}
+@title{Dust}
 
 @defmodule["dust.rkt" #:packages ()]
 
 This is where I put constants and helper functions that are needed pretty much everywhere in the
 project. In a simpler project these would go in @seclink["pollen-rkt"]{@filepath{pollen.rkt}} but
@@ -142,18 +143,26 @@
 (first-words txs-parens-commas 5)
 (first-words txs-short 5)
 ]
 
 @section{Article parsers and helpers}
+
+@defparam[listing-context ctxt (or/c 'blog 'feed 'print "") #:value ""]
+
+A parameter specifying the current context where any listings of articles would appear. Its purpose
+is to allow articles to exclude themselves from certain special collections (e.g., the blog, the RSS
+feed, print editions). Any article whose @code{conceal} meta matches the current context will not be
+included in any listings returned by the listing functions in
+@seclink["cache-rkt"]{@filepath{cache.rkt}}.
 
 @defproc[(default-title [body-txprs (listof txexpr?)]) string?]
 
 Given a list of tagged X-expressions (the elements of an article’s doc, e.g.), returns a string
 containing a suitable title for the document. (Uses @racket[first-words].)
 
-Titles are not required for articles, but there are contexts where you need something that
-serves as a title if one is not present, and that’s what this function supplies.
+Titles are not required for articles, but there are contexts where you need something that serves as
+a title if one is not present, and that’s what this function supplies.
 
 @examples[#:eval dust-eval
 (define doc 
   '(root (p "If I had been astonished at first catching a glimpse of so outlandish an "
             "individual as Queequeg circulating among the polite society of a civilized "

Index: code-docs/main.scrbl
==================================================================
--- code-docs/main.scrbl
+++ code-docs/main.scrbl
@@ -32,6 +32,7 @@
 @include-section["tour.scrbl"]
 @include-section["overview.scrbl"]
 @include-section["pollen.scrbl"]  @; pollen.rkt
 @include-section["dust.scrbl"]    @; dust.rkt
 @include-section["snippets-html.scrbl"] @; you get the idea
+@include-section["cache.scrbl"]
 @include-section["crystalize.scrbl"]

Index: code-docs/pollen.scrbl
==================================================================
--- code-docs/pollen.scrbl
+++ code-docs/pollen.scrbl
@@ -14,63 +14,21 @@
                      pollen/tag
                      pollen/setup
                      pollen/core
                      sugar/coerce))
 
-@title[#:tag "pollen-rkt"]{@filepath{pollen.rkt}}
+@title[#:tag "pollen-rkt"]{Pollen}
 
 @defmodule["pollen.rkt" #:packages ()]
 
 The file @filepath{pollen.rkt} is implicitly @code{require}d in every template and every @code{#lang
 pollen} file in the project. It defines the markup for all Pollen documents, and also re-provides
-everything provided by @code{crystalize.rkt}.
+everything provided by @filepath{cache.rkt} and @filepath{crystalize.rkt}.
 
 The @code{setup} module towards the top of the file is used as described in
 @racketmodname[pollen/setup].
 
-@section{Defining new tags}
-
-I use a couple of macros to define tag functions that automatically branch into other functions
-depending on the current output target format. This allows me to put the format-specific tag
-functions in separate files that have separate places in the dependency chain. So if only the HTML
-tag functions have changed and not those for PDF, the makefile can ensure only the HTML files are
-rebuilt.
-
-@defproc[#:kind "syntax"
- (poly-branch-tag (tag-id symbol?))
- (-> txexpr?)]
-
-Defines a new function @racket[_tag-id] which will automatically pass all of its arguments to a
-function whose name is the value returned by @racket[current-poly-target], followed by a hyphen,
-followed by @racket[_tag]. So whenever the current output format is @racket['html], the function
-defined by @racket[(poly-branch-tag _p)] will branch to a function named @racket[html-p]; when the
-current format is @racket['pdf], it will branch to @racket[pdf-p], and so forth.
-
-You @emph{must} define these branch functions separately, and you must define one for @emph{every}
-output format included in the definition of @racket[poly-targets] in this file’s @racket[setup]
-submodule. If you do not, you will get “unbound identifier” errors at expansion time.
-
-The convention in this project is to define and provide these branch functions in separate files:
-see, e.g., @filepath{tags-html.rkt}.
-
-Functions defined with this macro @emph{do not} accept keyword arguments. If you need keyword
-arguments, see @racket[poly-branch-kwargs-tag].
-
-@margin-note{The thought behind having two macros so similar is that, by cutting out handling for keyword
-arguments, @racket[poly-branch-tag] could produce simpler and faster code. I have not verified if
-this intuition is meaningful or correct.}
-
-@defproc[#:kind "syntax"
-         (poly-branch-kwargs-tag (tag-id symbol?))
-         (-> txexpr?)]
-
-Works just like @racket[poly-branch-tag], but uses Pollen’s @racket[define-tag-function] so that
-keyword arguments will automatically be parsed as X-expression attributes.
-
-Additionally, the branch functions called from the new function must accept exactly two arguments:
-a list of attributes and a list of elements.
-
 @section{Markup reference}
 
 These are the tags that can be used in any of @italic{The Local Yarn}’s Pollen documents (articles,
 etc).
 
@@ -295,5 +253,47 @@
 ◊for/s[x '(7 8 9)]{Now once for number ◊x}
 
 ◊;Above line is shorthand for this one:
 ◊for/splice[[(x (in-list '(7 8 9)))]]{Now once for number ◊x}
 }|
+
+@section{Defining new tags}
+
+I use a couple of macros to define tag functions that automatically branch into other functions
+depending on the current output target format. This allows me to put the format-specific tag
+functions in separate files that have separate places in the dependency chain. So if only the HTML
+tag functions have changed and not those for PDF, the @filepath{makefile} can ensure only the HTML files are
+rebuilt.
+
+@defproc[#:kind "syntax"
+ (poly-branch-tag (tag-id symbol?))
+ (-> txexpr?)]
+
+Defines a new function @racket[_tag-id] which will automatically pass all of its arguments to a
+function whose name is the value returned by @racket[current-poly-target], followed by a hyphen,
+followed by @racket[_tag]. So whenever the current output format is @racket['html], the function
+defined by @racket[(poly-branch-tag _p)] will branch to a function named @racket[html-p]; when the
+current format is @racket['pdf], it will branch to @racket[pdf-p], and so forth.
+
+You @emph{must} define these branch functions separately, and you must define one for @emph{every}
+output format included in the definition of @racket[poly-targets] in this file’s @racket[setup]
+submodule. If you do not, you will get “unbound identifier” errors at expansion time.
+
+The convention in this project is to define and provide these branch functions in separate files:
+see, e.g., @filepath{tags-html.rkt}.
+
+Functions defined with this macro @emph{do not} accept keyword arguments. If you need keyword
+arguments, see @racket[poly-branch-kwargs-tag].
+
+@margin-note{The thought behind having two macros so similar is that, by cutting out handling for keyword
+arguments, @racket[poly-branch-tag] could produce simpler and faster code. I have not verified if
+this intuition is meaningful or correct.}
+
+@defproc[#:kind "syntax"
+         (poly-branch-kwargs-tag (tag-id symbol?))
+         (-> txexpr?)]
+
+Works just like @racket[poly-branch-tag], but uses Pollen’s @racket[define-tag-function] so that
+keyword arguments will automatically be parsed as X-expression attributes.
+
+Additionally, the branch functions called from the new function must accept exactly two arguments:
+a list of attributes and a list of elements.