Tags are stored as search words

What you'll see

Tags come back in a different shape than they went in, and nothing anywhere reports an error.

• docly.addTag(path, "Annual report") and a later docly.getTags(path) returns two entries, not one.
• A tag typed as Faktura in the editor is listed as faktura everywhere else.
• docly.addTag(path, "a") stores nothing at all. The call succeeds, the returned list is unchanged, and there is no error to catch.
• docly.listFiles(path, { tag: "x" }) returns an empty array in a folder that plainly has tagged files in it.
• A tag is visible on the document but a full text search for the same word does not find it — usually after tags were written by something other than Docly.

What's actually happening

Tags and the full text index share their storage. A tag is a row linking a document to a word in the SearchWord table — the same table the indexer fills. That is what makes a tag searchable at all, and it is the reason a tag cannot be stored verbatim: a word written in a form the indexer never produces is a word the search can never match.

So every tag goes through the same normalization the indexer uses, whichever door it comes in by — the tag field in the editor, docly.addTag, or the tag filter on listFiles and getFiles. The rule is:

• Lower cased. Faktura and FAKTURA are the same tag as faktura.
• Split into words. "Annual report 2024" is not one tag with spaces in it. It is three tags.
• Words of one character are discarded, as is anything that carries no word at all. "a" and "!!" normalize to nothing.
• Duplicates collapse. Adding faktura to a document that already has it changes nothing and is not an error.

The consequence worth internalising is that the tag you asked for and the tag that exists are two different things, and only the second one is real. That is why addTag and removeTag hand back the document's full tag list rather than true: the list is the only place the difference shows up. Nothing throws when a tag evaporates.

The same rule applies on the way out. The tag filter normalizes its argument before it queries, which is why case does not matter there either. Two further consequences fall out of that:

• A filter of several words requires all of them. { tag: "Annual report" } matches only documents carrying both annual and report — which is exactly the set that would have been produced by adding that same string as a tag.
• A filter that normalizes to nothing returns an empty result, not an unfiltered one. Treating an unmatchable filter as "no filter" would hand back every file in the folder, which is the opposite of what the call asked for.

Tag writes are incremental — addTag only inserts the rows that are missing and removeTag only deletes the ones named — so two scripts tagging the same document at the same time cannot delete each other's work. A read-modify-write of the whole tag list can, and that is the pattern to avoid.

What to do

Prefer single, lower case words as tags. They survive normalization unchanged, so what you write is what you can later filter on. If a concept genuinely needs two words, accept that it is stored as two tags and filter on both.

Read the return value instead of assuming. Both write functions give you the resulting list, so verification costs nothing:

var tags = docly.addTag("/Invoices/Invoice 1001", userInput);
if (tags.indexOf(userInput.toLowerCase()) < 0) {
    // the input normalized into something else, or into nothing at all
}

Never tag straight from user input without checking the result. A one-character tag is silently dropped; a two-word tag silently becomes two. Neither raises an error, so an unchecked write looks successful right up until someone filters on the tag and gets nothing back.

When a tag filter returns an empty list, suspect the filter before the data. Call docly.getTags on a file you know is tagged and compare it with what you passed to the filter — the stored form is authoritative.

#{ docly.setMime("text/plain"); }#
stored:   #{ write(JSON.stringify(docly.getTags("/Invoices/Invoice 1001"))); }#
filtered: #{ write(docly.listFiles("/Invoices", { tag: "Invoice" }).length); }#

See Use a scratch hash file to test Docly functions, and delete the file when you are done.

Add and remove single tags rather than rewriting the list. addTag and removeTag touch only the tags they name. Reading every tag, changing the array and writing it all back will drop whatever another script added in between.

Do not write tag rows into the database directly. Anything that bypasses the normalization puts a word into the shared table in a form the search cannot reach.