◊(Local Yarn Code "Check-in [2cd87113]")

#lang scribble/manual

@; SPDX-License-Identifier: BlueOak-1.0.0
@; This file is licensed under the Blue Oak Model License 1.0.0.

@(require "scribble-helpers.rkt" scribble/example)

@(require (for-label deta
                     db
                     racket/base
                     racket/contract
                     sugar/coerce
                     pollen/template
                     "../dust.rkt"
                     "../crystalize.rkt"
                     "../cache.rkt"))

@(define example-eval (make-base-eval))
@(example-eval '(require "cache.rkt" txexpr))

@title[#:tag "cache-rkt"]{Cache}

@defmodule["cache.rkt" #:packages ()]

In this project there are several places – the blog, the footer on each page, the RSS feed, series
pages — where data from an amorphous group of Pollen documents is needed. This is what the cache is
for.

This module defines and provides the schema and database connection to the SQLite cache, and some
functions for retrieving records from the cache. Use these when you need quick access to pre-cooked
HTML.

@section{Cache database}

@defthing[cache-conn connection?]{
The database connection.
}

@defproc[(init-cache-db!) void?]{

Creates and initializes the SQLite database cache file (named @filepath{vitreous.sqlite} and located
in the project root folder) by running queries to create tables in the database if they do not
exist.

This function is called automatically in @seclink["pollen-rkt"]{@filepath{pollen.rkt}} whenever HTML
is the target output.

}

@defparam[current-plain-title title-plain non-empty-string? #:value "void"]{

Contains (or sets) the “plain” title (i.e., with no HTML markup) for the current article based on
analysis done by @racket[parse-and-cache-article!]. If the article did not specify a title,
a default title is supplied. If the article contained a @racket[note] that used the
@code{#:disposition} attribute, the disposition-mark may be included in the title.

This is a weird parameter, and at some point I will probably get rid of it and have
@racket[parse-and-cache-article!] supply it as an extra return value instead.

@margin-note{Note that this needs to be called @emph{after} @racket[parse-and-cache-article!] in
order to get an up-to-date value.}

}

@section{Retrieving cached data}

Some of this looks a little wacky, but it’s a case of putting a little extra complextity into the
back end to make things simple on the front end. These functions are most commonly used inside the
@emph{body} of a Pollen document (i.e., series pages). 

@filebox["series/my-series.poly.pm"
@codeblock|{
#lang pollen

◊title{My New Series}

...some other content

◊(<listing-excerpt> articles+notes #:order 'asc)
}|
]

@deftogether[(@defproc[(<listing-full> 
                                 [query-func (-> any/c query?)]
                                 [#:series series (or/c string? (listof string?) boolean?) #t]
                                 [#:limit limit integer? -1]
                                 [order stringish? 'desc]) txexpr?]
              @defproc[(<listing-excerpt>
                                 [query-func (-> any/c query?)]
                                 [#:series series (or/c string? (listof string?) boolean?) #t]
                                 [#:limit limit integer? -1]
                                 [order stringish? 'desc]) txexpr?]
              @defproc[(<listing-short>
                                 [query-func (-> any/c query?)]
                                 [#:series series (or/c string? (listof string?) boolean?) #t]
                                 [#:limit limit integer? -1]
                                 [order stringish? 'desc]) txexpr?])]{

Fetches the HTML for items from the SQLite cache, concatenates their HTML strings and returns
a @racket['style] tagged X-expression with this string as its element. The items will be ordered by
publish date according to @racket[_order] and optionally limited to the series specified in
@racket[_series].

The @racket[_query-func] should be either @racket[articles], which will create a listing of articles
only, or @racket[articles+notes], which will include notes intermingled with articles.

@margin-note{Note that the signature shown for the @racket[_query-func] argument above is
incomplete. If you choose to pass a function other than @racket[articles] or
@racket[articles+notes], you must use a function with exactly the same signature as those
functions.}

If @racket[_series] expression evaluates to @racket[#f], articles will not be filtered by series. If
it evaluates to @racket[#t] (the default), articles will be filtered by those that specify the
current output of @racket[here-output-path] in their @tt{series_pagenode} column in the SQLite
cache. If a string is supplied, articles will be filtered by those containing that exact value in
their @tt{series_pagenode} column in the SQLite cache.

The @racket[_order] expression must evaluate to either @racket["ASC"] or @racket["DESC"] and the
@racket[_limit] expressions must evaluate to a value suitable for use in the @tt{LIMIT} clause of
@ext-link["https://sqlite.org/lang_select.html"]{a SQLite @tt{SELECT} statement}. An expression that
evaluates to a negative integer (the default) is the same as having no limit.

The reason for enclosing the results in a @racket['style] txexpr is to prevent the HTML from being
escaped by @racket[->html] in the template. This tag was picked for the job because there will
generally never be a need to include any actual CSS information inside a @tt{<style>} tag in any
page, so it can be safely filtered out later. To remove the enclosing @tt{<style>} tag, see
@racket[unfence].

}

@defproc[(listing-htmls [listing-query query?]) (listof string?)]{

Returns the HTML bodies for the articles and/or notes returned by @racket[_listing-query] as a list
of strings.

}

@deftogether[(@defproc[(articles [type (or/c 'full 'excerpt 'short)]
                                 [#:series series (or/c string? (listof string?) boolean?) #t]
                                 [#:limit limit integer? -1]
                                 [order stringish? 'desc]) query?]
              @defproc[(articles+notes [type (or/c 'full 'excerpt 'short)]
                                       [#:series series (or/c string? (listof string?) boolean?) #t]
                                       [#:limit limit integer? -1]
                                       [order stringish? 'desc]) query?])]{

Create a query that fetches either articles only (@racket[articles]) or articles and notes
intermingled (@racket[articles+notes]), sorted by publish date and optionally limited to
a particular series.

Typically you will pass these functions by name to listing functions like @racket[<listing-full>]
rather than calling them directly.

@examples[#:eval example-eval
(articles 'full)]
}

@defproc[(unfence [html string?]) string?]{

Returns @racket[_html] with all occurrences of @racket["<style>"] and @racket["</style>"] removed.
The contents of the style tags are left intact.

Use this in templates with strings returned from @racket[->html] when called on docs that use the
@racket[<listing-full>] tag function or its siblings.

}

@defproc[(series-grouped-list) (listof (listof cache:series?))]{

Return a list of lists of all @racket[cache:series] in the cache database. The series are grouped so
that series using the same value in the @tt{noun-plural} column appear together.

}

@section{Deleting records}

@deftogether[(@defproc[(delete-article! [page stringish?]) void?]
              @defproc[(delete-notes!   [page stringish?]) void?])]{
              
Delete a particular article, or all notes for a particular article, respectively.

}

@section{Schema}

The cache database has four tables: @tt{articles}, @tt{notes}, @tt{index_entries} and @tt{series}. Each of these has a corresponding schema, shown below. In addition, there is a “virtual” schema, @tt{listing}, for use with queries which may or may not combine articles and notes intermingled.

The work of picking apart an article’s exported @tt{doc} and @tt{metas} into rows in these tables is done by @racket[parse-and-cache-article!].

The below are shown as @code{struct} forms but are actually defined with deta’s @racket[define-schema]. Each schema has an associated struct with the same name and a smart constructor called @tt{make-@emph{id}}. The struct’s “dumb” constructor is hidden so that invalid entities cannot be created. For every defined field there is an associated functional setter and updater named @tt{set-@emph{id}-field} and @tt{update-@emph{id}-field}, respectively.

@defstruct*[cache:article ([id                   id/f]
                           [page                 symbol/f]
                           [title-plain          string/f]
                           [title-html-flow      string/f]
                           [title-specified      boolean/f]
                           [published            string/f]
                           [updated              string/f]
                           [author               string/f]
                           [conceal              string/f]
                           [series-page          string/f]
                           [noun-singular        string/f]
                           [note-count           integer/f]
                           [doc-html             string/f]
                           [disposition          string/f]
                           [disp-html-anchor     string/f]
                           [listing-full-html    string/f]
                           [listing-excerpt-html string/f]
                           [listing-short-html   string/f])
            #:constructor-name make-cache:article]{

Table holding cached article information.

}

@defstruct*[cache:note ([id              id/f]
                        [page            symbol/f]
                        [html-anchor     string/f]
                        [title-html-flow string/f]
                        [title-plain     string/f]
                        [author          string/f]
                        [author-url      string/f]
                        [published       string/f]
                        [disposition     string/f]
                        [content-html    string/f]
                        [series-page     symbol/f]
                        [conceal         string/f]
                        [listing-full-html    string/f]
                        [listing-excerpt-html string/f]
                        [listing-short-html   string/f])
                       #:constructor-name make-cache:note]{

Table holding cached information on notes.

}

@defstruct*[cache:series ([id            id/f]
                          [page          symbol/f]
                          [title         string/f]
                          [published     string/f]
                          [noun-plural   string/f]
                          [noun-singular string/f])
                         #:constructor-name make-cache:series]{

Table holding cached information on series.

}

@defstruct*[cache:index-entry ([id id/f]
                               [entry       string/f]
                               [subentry    string/f]
                               [page        symbol/f]
                               [html-anchor string/f])
                              #:constructor-name make-cache:index-entry]{
                              
Table holding cached information about index entries found in articles.

}

@defstruct*[listing ([html        string/f]
                     [published   date/f]
                     [series-page symbol/f])
                    #:constructor-name make-listing]{

This is not a table that persists in the cache database; rather it is the schema targeted by
@racket[articles] and @racket[articles+notes] using deta’s @racket[project-onto].

}

#lang scribble/manual

@; SPDX-License-Identifier: BlueOak-1.0.0
@; This file is licensed under the Blue Oak Model License 1.0.0.

@(require "scribble-helpers.rkt")

@(require (for-label "../pollen.rkt"

                     "../crystalize.rkt"
                     "../cache.rkt"
                     racket/base
                     racket/contract
                     racket/string

                     txexpr

                     pollen/pagetree))


@title[#:tag "crystalize-rkt"]{Crystalize}

@defmodule["crystalize.rkt" #:packages ()]

“Crystalizing” is an extra layer in between docs and templates that destructures the @tt{doc} and
stores it in various pieces in a SQLite cache. Individual articles save chunks of rendered HTML to
the cache when their individual pages are rendered. When pulling together listings of articles in
different contexts that need to be filtered and sorted, a SQL query is much faster than trolling
through the Pollen cache for matching docs and regenerating the HTML.


@margin-note{These functions are designed to be used within the template for articles and series,



respectively, so that the cache is updated precisely when the web page is rendered.}


@defproc[(parse-and-cache-article! [pagenode pagenode?] [doc txexpr?]) non-empty-string?]{

Returns a string containing the HTML of @racket[_doc]. 


Privately, it does a lot of other work. The article is analyzed, additional metadata is constructed
and saved to the SQLite cache and saved using @racket[make-cache:article]. If the article specifies
a @racket['series] meta, information about that series is fetched and used in the rendering of the
article. If there are @racket[note]s or @racket[index] tags in the doc, they are parsed and saved
individually to the SQLite cache (using @racket[make-cache:note] and @racket[make-cache:index-entry]
respectively). If any of the notes use the @code{#:disposition} attribute, information about the
disposition is parsed out and used in the rendering of the article.
}






































@defproc[(cache-series!) void?]{














Attempts to look up certain values in @racket[current-metas] which we expect to be defined on



a typical series page, and saves them to the cache using @racket[make-cache:series]. If @tt{title}


is not defined in the current metas, you’ll get an error. If any of the others are missing, an empty

string is used.


}

.fileblock .SCodeFlow {
    padding-top: 0.7em;
    margin-top: 0;
}

.fileblock {
    width: 90%;
}

.fileblock_filetitle{
    background: #eee;
    text-align:right;
    padding: 0.15em;
    border: 1px dotted black;
    border-bottom: none;
}

.terminal, .browser {
    margin-bottom: 1em;
    padding: 0.5em;
    width: 88%;
    background: #fcfcfc;
    color: rgb(150, 35, 105);
}

.terminal .SIntrapara, .browser .SIntrapara, .fileblock .SIntrapara {
    margin: 0 0 0 0;
}

#lang scribble/manual

@; SPDX-License-Identifier: BlueOak-1.0.0
@; This file is licensed under the Blue Oak Model License 1.0.0.

@(require "scribble-helpers.rkt"
          (for-label "../pollen.rkt"))

@(require (for-label racket/base))

@title{Basic Notions}

@section[#:tag "design-goals"]{Design Goals}

The design of @italic{The Local Yarn} is guided by requirements that have evolved since I started
the site in 1999. I enumerate them here because they explain why the code is necessarily more
complicated than a typical blog:

@itemlist[
 @item{@bold{The writing will publish to two places from the same source: the web server, and the
 bookshelf.} The web server, because it’s a fun, fast way to publish writing and code to the whole
 world (you knew that already); but also on bookshelves, because
 @ext-link["https://thelocalyarn.com/excursus/secretary/posts/web-books.html"]{a web server is like
 a projector}, and I want to be able to turn it off someday and still have something to show for all
 my work. Plus, I just like printed books.}

 @item{@bold{Changes are part of the content.} I like to revisit, resurface and amend things I’ve
 written before. Views change, new ideas come along. In a typical blog the focus is always at
 whatever’s happening at the head of the time stream; an addendum to an older post is, for all
 practical purposes, invisible and nearly useless. I want every published edit to an article to be
 findable and linkable. I want addenda to be extremely visible. These addenda should also be able to
 mark major shifts in the author’s own perspective on what they originally wrote.}

 @item{@bold{Experimentation must be accomodated gracefully.} I should be able to write fiction,
 poetry, opinion pieces, minor observations or collections, or anything else, and have or create
 a good home for it here. Where dissimilar writings appear together, there should be signals that
 help the reader understand what they are looking at, switch contexts, and find more if they wish.}
 
 @item{@bold{Everything produced here should look good.}}

 @item{@bold{Reward exploration without disorienting the reader.}}
 
 @item{@bold{Everything produced here should be the result of an automatable process.} No clicking
 around to publish web pages and books.}
 
 ]

@section{Names for things and how they fit together}

The Local Yarn is mostly comprised of @tech{articles} (individual writings) which may contain
@tech{notes} (addenda by the author or others) and may also be grouped into @tech{series}. These are
similar to a typical blog’s @italic{posts}, @italic{comments} and @italic{categories}, but there are
important differences.

@subsection{Articles}

The @deftech{article} is the basic unit of content, like a typical blog post. In the web edition,
each article has its own @tt{.html} file; in print editions, an article may comprise either
a chapter or a part of a chapter, depending on the content.

An article can start out very small — just a date and a few sentences. Supplying a title is
optional. Later, it may grow in any of several directions: @tech{notes} can be added, or a title, or
cross-references to later articles; or it may be added to a series. Or it may just remain the way it
started.

@subsection{Notes}

A @deftech{note} is a comment or addendum to an @tech{article} using the @racket[note] tag. It may
be written by the same person who wrote the article, or submitted by a reader.

A note appears at the bottom of the article to which it is attached, but it also appears in the blog
and in the RSS feed as a separate piece of content, and is given the same visual weight as actual
articles.

A note may optionally have a @deftech{disposition} which reflects a change in attitude towards its
parent article. A disposition consists of a @italic{disposition mark} such as an asterisk or dagger,
and a past-tense verb. For example, an author may revisit an opinion piece written years earlier and
add a note describing how their opinion has changed; the tag for this note might include
@racket[#:disposition "* recanted"] as an attribute. This would cause the @tt{*} to be added to the
article’s title, and the phrase “Now considered recanted” to be added to the margin, with a link to
the note.

@subsubsection{Notes vs. blog “comments”}

Typical blog comments serve as kind of a temporary discussion spot for a few days or weeks after
a post is published. Commenting on an old post feels useless because the comment is only visible at
the bottom of its parent post, and older posts are never “bumped” back into visibility.

By contrast, notes on @italic{The Local Yarn} appear as self-contained writings at the top of the
blog and RSS feed as soon as they are published. This “resurfaces” the original article
to which they are attached. This extra visibility also makes them a good tool for the original
author to fill out or update the article. In effect, with notes, each article potentially becomes
its own miniature blog.

The flip side of this change is that what used to be the “comment section” is no longer allowed to
function as a kind of per-article chat.

@tabular[#:sep @hspace[1]
         #:style 'boxed
         #:row-properties '((bottom-border top))
         (list
          (list @bold{Typical Blog Comments} @bold{Local Yarn @emph{Notes}})
          (list "Rarely used after a post has aged"
                "Commonly used on posts many years old")
          (list "Visible only at the bottom of the parent post"
                "Included in the main stream of posts and in the RSS feed alongside actual posts")
          (list "Invites any and all feedback, from small compliments to lengthy rebuttals"
                "Readers invited to treat their responses as submissions to a publication.")
          (list "Usually used by readers"
                "Usually used by the original author")
          (list "Don’t affect the original post"
                "May have properties (e.g. disposition) that change the status and
presentation of the original post")
          (list "Moderation (if done) is typically binary: approved or not"
                "Moderation may take the form of edits and inline responses."))]

@subsection{Series}

A @deftech{series} is a grouping of @tech{articles} into a particular order under a heading.
A series may present its own written content alongside the listing of its articles.

The page for a series can choose how to display its articles: chronologically, or in an arbitrary
order. It can display articles only, or a mixed listing of articles and notes, like the blog. And it
can choose to display articles in list form, or as excerpts, or in their entirety.

A series can specify @italic{nouns} to be applied to its articles.

@subsubsection{Series vs. blog “categories”}

Typical blogs are not very good at presenting content that may vary a lot in length and style. The
kind of writing I want to experiment with may change a lot from day to day, season to season, decade
to decade. I wanted a single system that could organize and present it all, in a thoughtful,
coherent way, rather than starting a new blog every time I wanted to try writing a different kind of
thing.

My solution to this was to enrich the idea of “categories”. Rather than being simply labels that you
slap on blog posts, they would be titled collections with their own unique content and way of
presenting articles and notes. In addition, they could pass down certain properties to the posts
they contain, that can be used to give signals to the reader about what they are looking at. 

@tabular[#:sep @hspace[1]
         #:style 'boxed
         #:row-properties '((bottom-border top))
         (list
          (list @bold{Typical Blog Categories/Tags} @bold{Local Yarn @emph{Series}})
          (list "Every article needs to have one"
                "Many or most articles won’t have one")
          (list "Named with a single word" 
                "Name with a descriptive title")
          (list "Has no content or properties of its own"
                "Has its own content and properties")
          (list "Broad in scope, few in number"	
                "Narrow in scope, many in number")
          (list "Selected to be relevant for use across the entire lifetime of the site"
                "Selected without reference to future creative direction; may be closed after only
                 a few articles"))]

#lang scribble/manual

@; SPDX-License-Identifier: BlueOak-1.0.0
@; This file is licensed under the Blue Oak Model License 1.0.0.

@(require "scribble-helpers.rkt"
          scribble/example)

@(require (for-label "../pollen.rkt"
                     "../dust.rkt"
                     "../cache.rkt"
                     racket/base
                     racket/contract
                     txexpr
                     sugar/coerce
                     pollen/tag
                     pollen/setup
                     pollen/pagetree
                     pollen/core))

@(define dust-eval (make-base-eval))
@(dust-eval '(require "dust.rkt" txexpr))

@title{Dust}

@defmodule["dust.rkt" #:packages ()]

This is where I put constants and helper functions that are needed pretty much everywhere in the
project. In a simpler project these would go in @seclink["pollen-rkt"]{@filepath{pollen.rkt}} but
here I have other modules sitting “behind” that one in the @tt{require} chain.

(first-words txs-punc-and-split-elems 5)
(first-words txs-dashes 5)
(first-words txs-parens-commas 5)
(first-words txs-short 5)
]

@section{Article parsers and helpers}

@defparam[listing-context ctxt (or/c 'blog 'feed 'print "") #:value ""]

A parameter specifying the current context where any listings of articles would appear. Its purpose
is to allow articles to exclude themselves from certain special collections (e.g., the blog, the RSS
feed, print editions). Any article whose @code{conceal} meta matches the current context will not be
included in any listings returned by the listing functions in
@seclink["cache-rkt"]{@filepath{cache.rkt}}.

@defproc[(default-title [body-txprs (listof txexpr?)]) string?]

Given a list of tagged X-expressions (the elements of an article’s doc, e.g.), returns a string
containing a suitable title for the document. (Uses @racket[first-words].)

Titles are not required for articles, but there are contexts where you need something that serves as
a title if one is not present, and that’s what this function supplies.

@examples[#:eval dust-eval
(define doc 
  '(root (p "If I had been astonished at first catching a glimpse of so outlandish an "
            "individual as Queequeg circulating among the polite society of a civilized "
            "town, that astonishment soon departed upon taking my first daylight "
            "stroll through the streets of New Bedford…")))

#lang scribble/manual

@; SPDX-License-Identifier: BlueOak-1.0.0
@; This file is licensed under the Blue Oak Model License 1.0.0.

@(require "scribble-helpers.rkt"
          racket/runtime-path
          (for-label racket/base
                     "../crystalize.rkt"))

@title{Local Yarn: Source Code Notes}

@author{Joel Dueck}

These are my notes about the internals of the Local Yarn source code. In other words, a personal
reference, rather than a tutorial. 



You’ll get the most out of these notes if you have read @other-doc['(lib
"pollen/scribblings/pollen.scrbl")], and worked through the tutorials there by hand.

@margin-note{Note that these pages are heavily interlinked with the central Racket documentation at
@tt{docs.racket-lang.org}, which are written and maintained by others. 

Some links from those pages will not work unless you @ext-link["#"]{open this page in its own tab}.
}

Here’s a rough diagram showing how the @tt{.rkt} modules in this project relate to each other, and
to the Pollen source documents. This is the least complex system I could devise that would @tt{A)}
implement everything I want in my @secref["design-goals"], @tt{B)} cleanly separate dependencies for
print and web output, and @tt{C)} organize an ever-growing collection of hundreds of individual
notes and articles without noticable loss of speed. 

@(define-runtime-path source-diagram "source-diagram.png")
@centered{@responsive-retina-image[source-diagram]}

The solid-line connections indicate explicit @racket[require] relationships. Dotted arrows generally
indicate implicit exports in the Pollen environment: docs and metas to templates,
@seclink["pollen-rkt"]{@filepath{pollen.rkt}} to source documents and templates. The orange lines
highlight the provision and use of the functions in
@seclink["crystalize-rkt"]{@filepath{crystalize.rkt}}.

Individual articles, while they are being rendered to HTML pages, save copies of their metadata and
HTML to the SQLite cache. This is done by calling @racket[parse-and-cache-article!] from within
their template. Likewise, series pages cache themselves with a call to @racket[cache-series!] from
within their template.

Any pages that gather content from multiple articles, such as Series pages and the RSS feed, pull
this content directly from the SQLite cache. This is much faster than trawling through Pollen’s
cached metas of every article looking for matching articles.

@local-table-of-contents[]

@include-section["tour.scrbl"]
@include-section["design.scrbl"]
@include-section["pollen.scrbl"]  @; pollen.rkt
@include-section["dust.scrbl"]    @; dust.rkt
@include-section["snippets-html.scrbl"] @; you get the idea
@include-section["cache.scrbl"]
@include-section["crystalize.scrbl"]

#lang scribble/manual

@; SPDX-License-Identifier: BlueOak-1.0.0
@; This file is licensed under the Blue Oak Model License 1.0.0.

@(require "scribble-helpers.rkt")
@(require (for-label "../pollen.rkt"
                     "../dust.rkt"
                     "../cache.rkt"
                     "../crystalize.rkt"
                     racket/base
                     racket/contract
                     racket/string
                     txexpr
                     pollen/tag
                     pollen/setup
                     pollen/core
                     sugar/coerce))

@title[#:tag "pollen-rkt"]{Pollen}

@defmodule["pollen.rkt" #:packages ()]

The file @filepath{pollen.rkt} is implicitly @code{require}d in every template and every @code{#lang
pollen} file in the project. It defines the markup for all Pollen documents, and also re-provides
everything provided by @seclink["cache-rkt"]{@filepath{cache.rkt}} and
@seclink["crystalize-rkt"]{@filepath{crystalize.rkt}}.

The @code{setup} module towards the top of the file is used as described in
@racketmodname[pollen/setup].











































@section{Markup reference}

These are the tags that can be used in any of @italic{The Local Yarn}’s Pollen documents (articles,
etc).

@defproc[(title [element xexpr?] ...) txexpr?]{

@margin-note{The @code{title} function is not actually defined in @filepath{pollen.rkt} or anywhere
else. In Pollen, any undefined function @tt{title} defaults to @racket[(default-tag-function
title)], which is what I want. It is documented here because its presence or absence has
side-effects on the display of the article.}

Supplies a title for the document. You can use any otherwise-valid markup within the title tag. 

Titles are optional; if you don’t specify a title, the article will appear without one. This is
a feature!
}

@defproc[(p [element xexpr?] ...) txexpr?]{

Wrap text in a paragraph. You almost never need to use this tag explicitly; 
just separate paragraphs by an empty line.

Single newlines within a paragraph will be replaced by spaces, allowing you to use
@ext-link["https://scott.mn/2014/02/21/semantic_linewrapping/"]{semantic line wrapping}.
}

@defproc[(newthought [element xexpr?] ...) txexpr?]{

An inline style intended for the first few words of the first paragraph in a new section. Applies
a “small caps” style to the text. Any paragraph containing a @code{newthought} tag is given extra
vertical leading.

Rule of thumb: within an article, use either @code{section}/@code{subsection} or @code{newthought}
to separate sections of text, but not both. Even better, keep it consistent across articles within
a series.

If you just need small caps without affecting the paragraph, use @racket[caps].
}

@deftogether[(@defproc[(section    [element xexpr?] ...) txexpr?]
              @defproc[(subsection [element xexpr?] ...) txexpr?])]{

Create second- and third-level headings, respectively. This is counting the article's title as the
first-level header (even if the current article has no title).

}

@defproc[(block [element xexpr?] ...) txexpr?]{

A container for content that should appear grouped together on larger displays. Intended for use in
Series pages, where the template is very minimal to allow for more customization. You would want
output from @racket[<listing-short>] to appear inside a @racket[block], but you would want output
from @racket[<listing-full>] to appear outside it (since each article effectively supplies its own
block). Only relevant to HTML output.

}

@deftogether[(@defproc[(link [link-id stringish?] [link-text xexpr?]) txexpr?]
              @defproc[(url  [link-id stringish?] [url string?]) void?])]{

All hyperlinks are specified reference-style. So, to link some text, use the @code{link} tag with
an identifier, which can be a string, symbol or number. Elsewhere in the text, use @code{url} with
the same identifier to specify the URL:

@codeblock|{
  #lang pollen
  If you need help, ◊link[1]{Google it}.

  ◊url[1]{https://google.com}
}|

The @code{url} tag for a given identifier may be placed anywhere in the document, even before it is
referenced. If you create a @code{link} for an identifier that has no corresponding @code{url},
a @code{"Missing reference: [link-id]"} message will be substituted for the URL. Conversely, 
creating a @code{url} that is never referenced will produce no output and no warnings or errors.

}

@deftogether[(@defproc[(figure     [image-file string?] [caption xexpr?] ...) txexpr?]
              @defproc[(figure-@2x [image-file string?] [caption xexpr?] ...) txexpr?])]{

Insert a block-level image. The @racket[_image-file] should be supplied as a filename only, with no
folder names. It is assumed that the image is located inside an @racket[images-folder] within the
same folder as the source document.

For web output, using @racket[figure-@2x] will produce an image hard-coded to display at half its
actual size, or the width of the text block, whichever is smaller.

}

@defproc[(image-link [image-file string?] [link-text xexpr?] ...) txexpr?]{

Adds a hyperlink to @racket[_image-file], supplied as a filename only with no folder names. It is
assumed that the image is located inside an @racket[images-folder] within the same folder as the
source document.

}

@deftogether[(@defproc[(fn    [fn-id stringish?]) txexpr?]
              @defproc[(fndef [fn-id stringish?] [elements xexpr?] ...) txexpr?])]{

As with hyperlinks, footnotes are specified reference-style. In the output, footnotes will be
numbered according to the order in which their identifiers are referenced in the source document.

Example:

@codeblock|{
  #lang pollen
  Shoeless Joe Jackson was one of the best players of all time◊fn[1].

  ◊fndef[1]{But he might have lost the 1919 World Series on purpose.}
}|

You can refer to a given footnote definition more than once.

The @code{fndef} for a given id may be placed anywhere in the source document, even before it is
referenced. If you create a @code{fn} reference without a corresponding @code{fndef},
a @code{"Missing footnote definition!"} message will be substituted for the footnote text.
Conversely, creating a @code{fndef} that is never referenced will produce no output, warning or
error.

}

@deftogether[(@defproc[(dialogue [elements xexpr?] ...) txexpr?]
              @defproc[(say [interlocutor string?] [elements xexpr?] ...) txexpr?]
              @defproc[(saylines [interlocutor string?] [elements xexpr?] ...) txexpr?])]{

Use these tags together for transcripts of dialogue, chats, screenplays, interviews and so
forth. The @racket[saylines] tag is the same as @racket[say] except that within @racket[saylines],
linebreaks within paragraphs are preserved.

Example usage:

@codeblock|{
  #lang pollen

  ◊dialogue{
    ◊say["Tavi"]{You also write fiction, or you used to. Do you still?}
    ◊say["Lorde"]{The thing is, when I write now, it comes out as songs.}
  }
}|

}

@defproc[(index [#:key key string? ""] [elements xexpr?] ...) txexpr?]{

Creates a bidirectional link between this spot in the document and an entry in the keyword index
under @racket[_key]. If @racket[_key] is not supplied, the string contents of @racket[_elements] are
used as the key. Use @tt{!} to split @racket[_key] into a main entry and a subentry.

The example below will create two index entries, one under the heading “compassion” and one under
the main heading "cats" and a subheading “stray”:

@codeblock|{
  #lang pollen

  “I have a theory which I suspect is rather immoral,” Smiley 
  went on, more lightly. “Each of us has only a quantum of 
  ◊index{compassion}. That if we lavish our concern on every
  stray ◊index[#:key "cats!stray"]{cat} we never get to the
  centre of things. What do you think of it?” 
}|

}

@defproc[(note [#:date date-str non-empty-string?]
               [#:author author string? ""]
               [#:author-url author-url string? ""]
               [#:disposition disp-str string? ""]) txexpr?]{

Add a @tech{note} to the “Further Notes” section of the article. 


The @code{#:date} attribute is required and must be of the form @tt{YYYY-MM-DD}.

The @code{#:author} and @code{#:author-url} attributes can be used to credit notes from other
people. If the @code{#:author} attribute is not supplied then the value of @code{default-authorname}
is used.

Some caveats (for now):

@itemlist[
  @item{Avoid defining new footnotes using @code{fndef} inside a @code{note}; these footnotes will
  be placed into the main footnote section of the article, which is probably not what you want.}
]

}

@defproc[(verse [#:title title string? ""] [#:italic? italic boolean? #f] [element xexpr?] ...)
         txexpr?]{

Typeset contents as poetry, with line breaks preserved and the block centered on the longest line.
To set the whole block in italic, use @code{#:italic? #t} — otherwise, use @code{i} within the
block.

If the first element in an article is a @racket[verse] tag with the @racket[#:title] attribute
specified, that title is used as the article’s title if the normal @racket[title] tag is absent.

}

@defproc[(blockquote [element xexpr?] ...) txexpr?]{

Surrounds a block quotation. To cite a source, include a @code{footer} tag at the bottom.

}

@defproc[(blockcode [element xexpr?] ...) txexpr?]{

Typeset contents as a block of code using a monospace font. Line breaks are preserved.

}

@deftogether[(@defproc[(i      [element xexpr?] ...) txexpr?]
              @defproc[(em     [element xexpr?] ...) txexpr?]
              @defproc[(b      [element xexpr?] ...) txexpr?]
              @defproc[(strong [element xexpr?] ...) txexpr?]
              @defproc[(strike [element xexpr?] ...) txexpr?]
              @defproc[(ol     [element xexpr?] ...) txexpr?]
              @defproc[(ul     [element xexpr?] ...) txexpr?]
              @defproc[(item   [element xexpr?] ...) txexpr?]
              @defproc[(sup    [element xexpr?] ...) txexpr?]
              @defproc[(caps   [element xexpr?] ...) txexpr?]
              @defproc[(code   [element xexpr?] ...) txexpr?])]{

Work pretty much how you’d expect.

}

@section{Convenience macros}

@defform[(for/s thing-id listofthings result-exprs ...)
         #:contracts ([listofthings (listof any/c)])]{

A shorthand form for Pollen’s @code{for/splice} that uses far fewer brackets when you’re only
iterating through a single list.

@codeblock|{
#lang pollen

◊for/s[x '(7 8 9)]{Now once for number ◊x}

◊;Above line is shorthand for this one:
◊for/splice[[(x (in-list '(7 8 9)))]]{Now once for number ◊x}
}|

}

@section{Defining new tags}

I use a couple of macros to define tag functions that automatically branch into other functions
depending on the current output target format. This allows me to put the format-specific tag
functions in separate files that have separate places in the dependency chain. So if only the HTML
tag functions have changed and not those for PDF, the @filepath{makefile} can ensure only the HTML
files are rebuilt.

@defproc[#:kind "syntax"
 (poly-branch-tag (tag-id symbol?))
 (-> txexpr?)]{

Defines a new function @racket[_tag-id] which will automatically pass all of its arguments to a
function whose name is the value returned by @racket[current-poly-target], followed by a hyphen,
followed by @racket[_tag]. So whenever the current output format is @racket['html], the function
defined by @racket[(poly-branch-tag _p)] will branch to a function named @racket[html-p]; when the
current format is @racket['pdf], it will branch to @racket[pdf-p], and so forth.

You @emph{must} define these branch functions separately, and you must define one for @emph{every}
output format included in the definition of @racket[poly-targets] in this file’s @racket[setup]
submodule. If you do not, you will get “unbound identifier” errors at expansion time.

The convention in this project is to define and provide these branch functions in separate files:
see, e.g., @filepath{tags-html.rkt}.

Functions defined with this macro @emph{do not} accept keyword arguments. If you need keyword
arguments, see @racket[poly-branch-kwargs-tag].

@margin-note{The thought behind having two macros so similar is that, by cutting out handling for keyword
arguments, @racket[poly-branch-tag] could produce simpler and faster code. I have not verified if
this intuition is meaningful or correct.}

}

@defproc[#:kind "syntax"
         (poly-branch-kwargs-tag (tag-id symbol?))
         (-> txexpr?)]{

Works just like @racket[poly-branch-tag], but uses Pollen’s @racket[define-tag-function] so that
keyword arguments will automatically be parsed as X-expression attributes.

Additionally, the branch functions called from the new function must accept exactly two arguments:
a list of attributes and a list of elements.

}

#lang racket/base

; SPDX-License-Identifier: BlueOak-1.0.0
; This file is licensed under the Blue Oak Model License 1.0.0.

;; Convenience/helper functions for this project’s Scribble documentation

(require scribble/core
         scribble/manual/lang
         scribble/html-properties
         scribble/private/manual-sprop
         scribble/decode
         racket/runtime-path
         (only-in net/uri-codec uri-encode))
(provide (all-defined-out))

(define-runtime-path custom-css "custom.css")

(define repo-url/ "https://thelocalyarn.com/code/")

;; Link to a ticket on the Fossil repository by specifying the ticket ID.
;; The "_parent" target breaks out of the iframe used by the Fossil repo web UI.
(define (ticket id-str)
  (hyperlink (string-append repo-url/ "tktview?name=" id-str)
             "ticket "
             (tt id-str)

             #:style (style #f (list (attributes '((target . "_parent")))))))

(define (responsive-retina-image img-path)
  (image img-path 
         #:scale 0.5
         #:style (style #f (list (attributes '((style . "max-width:100%;height:auto;")))))))

;;
;; From https://github.com/mbutterick/pollen/blob/master/pollen/scribblings/mb-tools.rkt
;;

(define (terminal . args)
  (compound-paragraph (style "terminal" (list (css-style-addition custom-css) (alt-tag "div")))
                      (list (apply verbatim args))))

(define (cmd . args)
  (elem #:style (style #f (list (color-property "black"))) (tt args)))

(define (fileblock filename . inside)
  (compound-paragraph 
   (style "fileblock" (list* (alt-tag "div") 'multicommand
                             (box-mode "RfileboxBoxT" "RfileboxBoxC" "RfileboxBoxB") 
                             scheme-properties))
   (list
    (paragraph (style "fileblock_filetitle" (list* (alt-tag "div") (box-mode* "RfiletitleBox") scheme-properties))
               (list (make-element
                      (style "fileblock_filename" (list (css-style-addition custom-css)))
                      (if (string? filename)
                          (filepath filename)
                          filename))))
    (compound-paragraph 
     (style "fileblock_filecontent" (list* (alt-tag "div") (box-mode* "RfilecontentBox") scheme-properties))
     (decode-flow inside)))))

                     racket/contract
                     racket/string
                     pollen/template
                     pollen/pagetree
                     txexpr
                     sugar/coerce))

@title{HTML snippets}

@defmodule["snippets-html.rkt" #:packages ()]

Each “snippet” module provides all (well @emph{most of}) the document- and article-level blocks of
structural markup necessary for a particular target output format; this one is for HTML. The idea is
that any block of markup that might be reused across more than one template should be a function.

The functions in the snippets modules follow two conventions in this project:

@itemlist[
  @item{Functions that return strings of HTML have the prefix @tt{html$-}.}
  @item{Such functions do not do any parsing or destructuring of complex objects; every separate
  piece that will be inserted into the template is passed in as a separate argument. This makes it

    <p><b>◊|title|</b><p>
    ◊(->html body-tx)
  })
}

@section{HTML Snippet functions}

@defproc[(html$-page-head [title (or/c string? #f) #f] [close-head? boolean? #t])
non-empty-string?]{

Returns the @tt{<head>} section of an HTML document.

If @racket[_title] is a string it will be used inside the @tt{<title>} tag.

If you want to include additional stuff inside the @tt{<head>}, you can set @racket[_close-head?] to
@racket[#f] to prevent it from including the closing @tt{</head>} tag (you’ll have to add it
yourself).
}

@defproc[(html$-page-body-open [body-class string? ""]) non-empty-string?]{

Returns the opening @tt{<body>} and @tt{<main>} tags and elements that immediately follow, such as
site header, logo and navigation.

If @racket[_body-class] is a non-empty string, its contents will be included in the @tt{class}
attribute of the @tt{<body>} tag.
}

@defproc[(html$-series-list) non-empty-string?]{

Returns an HTML @tt{<section>} containing a list of all series, grouped by their “plural nouns”. The
grouped list will flow into columns on wider displays.
}

@defproc[(html$-article-open [pagenode pagenode?] 
                             [title-specified-in-doc? boolean?] 
                             [title txexpr?] 
                             [pubdate string?])
          non-empty-string?]{

Returns the opening @tt{<article>} tag and elements that immediately follow: permlink, publish date,
and opening @tt{<section>} tag.

The @racket[_title-specified-in-doc?] form changes the HTML markup structure used.
}

@defproc[(html$-article-close [footertext string?]) non-empty-string?]{

Returns a string containing a closing @tt{<section>} tag, a @tt{<footer>} element containing
@racket[_footertext], and a closing @tt{<article>} tag. If @racket[_footertext] is empty, the
@tt{<footer>} element will be omitted.
}

@defproc[(html$-article-listing-short [pagenode pagenode?] [pubdate string?] [title string?])
non-empty-string?]{

Returns a string of HTML for an article as it would appear in a listing context in “short” form
(date and title only, with link).
}

@defproc[(html$-page-footer) non-empty-string?]{
Returns an HTML @tt{<footer>} tag for use at the bottom of each page.
}

@defproc[(html$-page-body-close) non-empty-string?]{

Returns a string containing the page’s @tt{<footer>} and closing tags.
}

@defproc[(html$-note-contents [disposition-mark string?] [elems (listof xexpr?)])
non-empty-string?]{

Returns a string containing the body-elements of a note converted to HTML. If
@racket[_disposition-mark] is not empty, a @tt{<span>} containing it will be inserted as the first
element of the first block-level element.
}

@defproc[(html$-note-listing-full [pagenode pagenode?]
                                  [note-id string?] 
                                  [title-html-flow string?]
                                  [date string?]
                                  [contents string?] 
                                  [author string? (default-authorname)] 
                                  [author-url string? ""])
          non-empty-string?]{

Returns a string containing the complete HTML markup for a @racket[note], including title, permlink,
date, contents and author, suitable for display outside the context of its parent article.
}

@defproc[(html$-note-in-article [id string?] 
                                [date string?] 
                                [contents string?] 
                                [author string?]
                                [author-url string?]) non-empty-string?]{

Like @racket[html$-note-listing-full], but returns HTML for a @racket[note] suitable for display
inside its parent article.
}

@defproc[(html$-notes-section [note-htmls string?]) non-empty-string?]{

Returns the complete HTML for the @italic{Further Notes} section of an article.
}

@defproc[(html$-paginate-navlinks [current-page exact-positive-integer?] 
                                  [pagecount exact-positive-integer?]
                                  [basename string?]) string?]{

On the “blog”, the articles are split across multiple files: @filepath{blog-pg1.html},
@filepath{blog-pg2.html}, etc. This function provides a string containing HTML for a group of links
that can be given within each file, to link to the pages that come before/after it. 

The links are enclosed within @tt{<li>} tags. It’s up to the calling site to provide the enclosing
@tt{<ul>} tag (in case any custom styling or IDs need to be applied).
}

#lang scribble/manual

@; SPDX-License-Identifier: BlueOak-1.0.0
@; This file is licensed under the Blue Oak Model License 1.0.0.

@(require "scribble-helpers.rkt")

@(require (for-label racket/base pollen/core "../pollen.rkt"))

@title{How I Publish: A Quick Tour}

This isn’t a tutorial, since these steps probably won’t all work on your computer. Think of these
narrations like me talking while I drive.

@section{Creating an article}

Open a terminal window.

@terminal{@cmd{> cd /path/to/thelocalyarn}}

The @tt{make} command provides a high-level control panel for common tasks. Typing just make from
a terminal window shows a list of options:

@terminal{
@cmd{> make}
article     Start a new article from a template
help        Displays this help screen
publish     Sync all HTML and PDF stuff to the public web server
scribble    Rebuild code documentation and update Fossil repo
web         Rebuild all web content (not PDFs)
zap         Clear Pollen and Scribble cache, and remove all HTML output
}

Following the first option in this list, I type @tt{make article}, and enter a post title when
prompted:

@terminal{
@cmd{> make article}
racket -tm util/newpost.rkt
Enter title: @cmd{My New Post}
}

The script creates a new @filepath{.poly.pm} file using a normalized version of the title for the
filename, and opens it in my editor (this is currently hardcoded to use MacVim). When the file pops
up we see a basic template ready to edit:

@filebox["articles/my-new-post.poly.pm"
@codeblock|{
#lang pollen
 
◊; Copyright 2020 by Joel Dueck. All Rights Reserved.
◊(define-meta draft #t)
◊(define-meta published "2020-01-18")
 
◊title{My New Post}
 
Write here!
}|]

At this point I might delete the @tt{◊title} line, since specifying a formal title is optional
(other than the one needed to generate the filename). I might also add a @racket[define-meta] for
@tt{series} or @tt{topics}. 

As long as the @racket[define-meta] for @tt{draft} is @racket[#t], the new article will not appear
in the RSS feed, or in the blog or any series pages.

When satisfied with the post I’ll remove the @racket[define-meta] for @tt{draft}, save it one last
time, then go back to the terminal:

@terminal{
@cmd{> make web}
[lots of output: rebuilds blog pages, keyword index, RSS feed]
@cmd{> make publish}
[lots of output: uploads all web content to the server]
}

The article also needs to be added to the Fossil repository, so revisions to it will be tracked:

@terminal|{
|@cmd{> fossil add article/my-new-post.poly.pm}
ADDED  article/my-new-post.poly.pm

|@cmd{> fossil commit -m "Publish ‘My New Post’"}
Autosync:  https://joel@thelocalyarn.com/code
Round-trips: 2   Artifacts sent: 0  received: 1
Pull done, sent: 892  received: 8720  ip: 162.243.186.132
New_Version: 15507b62416716a3f0be3c444b0fc09aa4364b989140c5788cf679eb0b2463a6
Autosync:  https://joel@thelocalyarn.com/code
Round-trips: 2   Artifacts sent: 2  received: 1
Sync done, sent: 10153  received: 4680  ip: 162.243.186.132
}|

As you can see, Fossil does an automatic pull before the commit, and another automatic push
afterwards. This commit is now visible on the public timeline, and the source code for the article
can now be seen on the public repo at @tt{thelocalyarn.com/code/}.

@section{Adding notes to an article}

A few days (or years) after doing the above, I receive an email from Marjorie with commenting on
@italic{My New Post} and I decide to publish her comments.

I open the article in my editor and add some lines to the end:

@filebox["articles/my-new-post.poly.pm"
@codeblock|{
#lang pollen
 
◊; Copyright 2020 by Joel Dueck. All Rights Reserved.
◊(define-meta published "2020-01-18")
 
◊title{My New Post}
 
It’s a 4⨉4 treated. I got twenty, actually, for the backyard fence.

◊note[#:date "2020-01-23" #:author "Marjorie"]{
 Hope you sank that thing below the frost line.
}
}|]

This looks like a blog-style comment, but the @racket[note] tag function has some special powers
that typical comments don’t have, as we’ll see in a moment.

I save this, go back to the terminal and do @tt{make web} and @tt{make publish} as before.

Now if you open the article’s permlink, you’ll see the note appears in a “Further Notes” section at
the bottom — again, just like a normal blog post comment.

But if you go to the Blog section, you’ll see the note appearing in its own space right alongside
the other articles, as if it were a separate post. It will also appear in a separate entry in the
RSS feed.

@section{What’s not here yet}

Eventually there will be facilities for creating PDF files of individual articles, and print-ready
PDFs of books containing collections of articles. 

		--exclude=makefile 
	rm -rf ~/Desktop/publish
	fossil uv sync

scribble: ## Rebuild code documentation and update Fossil repo
	scribble --htmls +m --redirect https://docs.racket-lang.org/local-redirect/ code-docs/main.scrbl
	fossil uv rm scribbled/*
	mv scribbled/site-footer.html main/ || true
	rm -rf scribbled/*
	mv main/* scribbled/
	cp code-docs/scribble-iframe.html scribbled/scribble.html
	rm -rf main
	fossil uv add scribbled/*
	fossil uv sync