1/ General Points

User's Guide

• IndexMatic sounds easy to play but difficult to master. Is there a good step by step instruction on how to use its advanced features?

[BOTH VERSIONS] The right place to learn is the user's guide (PDF), which is provided in both English and French language. This manual has been designed to guide you from the basics to the next level, including technical information on the query interpreter, regular expressions, and additional notes about how the script actually scans and parses InDesign documents.

More concrete examples and case studies will be addressed in this FAQ, which I hope will be enriched by users' comments!

Version Number

• How to know what exact version of IndexMatic I'm using?

[BOTH VERSIONS] The major version number is displayed in the title bar of the main dialog, typically, “IndexMatic PRO | 2.0” (pro version) or “IndexMatic TRY | 2.0” (trial version). The full version number is displayed at the bottom right of the main dialog. E.g: “2.026”.

CS5.5 Compatibility

• We bought a license of IndexMatic PRO, but some days ago we also bought the upgrade to InDesign CS 5.5. Can we reuse the script in CS5.5?

[BOTH VERSIONS] IndexMatic2 works in both InDesign CS3, CS4, CS5, and CS5.5. (In CS3, a few features are not available.) To install the script in a newer InDesign installation, simply put the script file, IndexMaticPro.jsx or IndexMaticTry.jsx, in the Script Panels folder of the new release.

Reminder. — The easiest way to locate the Scripts Panel folder is to start InDesign, open the Scripts panel (Window > Utilities > Scripts), right-click the “User” or “Application” item, then click “Reveal in Finder/Explorer.”

Saving User's Preferences

• Is there any way to save the last-used state of the dialog box? It is a little frustrating having to choose each of the menus each time I use the script.

[TRY VERSION] InDesign session persistent settings are only supported in the PRO version of IndexMatic.

IndexBrutal vs. IndexMatic

• I used your script “IndexBrutal” for years. Can I reuse in IndexMatic the wordlists I prepared in IndexBrutal?

[BOTH VERSIONS] All what you did with IndexBrutal can be done with IndexMatic2 (and so much more!) but you cannot reuse IndexBrutal's queries “as is” if syntactic operators were invoked. Indeed, IndexMatic offers a different way to manage case sensitivity, topic-rewriting and the "whole word" option. Roughly, you should apply the following conversion rules:

IndexBrutal syntax	IndexMatic2 syntax
roar|noise	roar=>noise
>punish	punish/W
!smith	smith/I
!UNESCO	UNESCO/i

2/ Scope, Context & Search options

Preventing the index from being indexed!

• As I lay out the index at the end of my book, I don't want that IndexMatic scans the index chapter itself! How to tell the script to ignore that section?

[BOTH VERSIONS] There are several ways to proceed. I recommend one of the following methods:

(a) In the Scope panel, select Range > Page range... and enter the explicit interval of page numbers that you need to inspect, excluding the Index pages. E.g.: 1-250 if the index is inserted on page 251.

(b) Another easy strategy is to set up all text to be indexed on a dedicated layer, e.g. “Indexable”, while the Index section is set on another layer. So you don't have to worry about page ranges: in the Scope panel, just select “Indexable” in the Layer(s) dropdown list.

Autoextracting a Vocabulary

• I need to index an entire book but I have not preset any wordlist. Is there a way to automatically extract the most relevant vocab entries and then manually refine the set before processing IndexMatic?

[PRO VERSION] IndexMatic offers an “Automatic Mode” which can be used to detect and grab the most representative words from your document(s). First, open your book in InDesign and launch the script. In the Scope panel, select the “Book” item in the Document(s) list. In the Default Options panel, set the Page Rank to 3 or 4 (these are usually appropriate values). Click the Hits... button to open the Hit Report dialog. Uncheck “Display stats” and click Run. This creates a wordlist that you can easily cleanup and use as a starting point in the Query List mode.

Matches and Page Rank

• Given a regular expression like /Steve|Bill/ which produces two terms—"Steve" and "Bill"—does the Page Rank apply on each term, or considering the total number of matches per page?

[BOTH VERSIONS] The Page Rank operates on each separate term that the query implicitly produces. Suppose that "Steve" occurs three times on page 10 and two times on page 11, while "Bill" occurs two times on page 10 and three times on page 11:

    On Page 10: … Steve … Bill … Steve … Bill … Steve.
    On Page 11: … Bill … Steve … Bill … Steve … Bill.

Then, the query:

    /Steve|Bill/3

sets the Page Rank to 3 and leads to the following output:

    Bill: 11
    Steve: 10

By contrast, if the query provides an explicit term:

    /Steve|Bill/3 => My Friends

then IndexMatic considers that each alternate match counts in the "My Friends" Page Rank. So the above query leads to:

    My Friends: 10-11

(Since the total match count is 5 on each page, the Page Rank is obviously satisfied.)

“Whole Word” and Regular Expressions

• Our document relies on a special markup syntax using the following format:
"...\index{the word to be indexed}..."
As we want to extract every marked word, we tried the query:
/\\index\{([^}]+)\}/ => $1, but this does not work.

[BOTH VERSIONS] Always consider the context where the matches appear. Your target format, \index{the word to be indexed}, seems to be directly embedded in the text—I mean, without any separation—so you probably have to disable the Whole Word option. Try one of the following:

(a) Uncheck the “Whole Word” checkbox in the Default Options panel (so that the flag is globally disabled).

    OR

(b) Append the local flag W at the end of your pattern:
    /\\index\{([^}]+)\}/W => $1

Conditional Text

• Our annual report is written in several languages that we manage in a single InDesign document through conditional texts (rather than linguistic layers). Can IndexMatic target separately each language (i.e., each condition)?

[BOTH VERSIONS] Regarding conditional text, IndexMatic simply considers the contents wich is enabled at the moment you launch the script. So, just activate the condition that corresponds to a specific language and you get the related index.

Targeting several Character Styles

• The data we have to index are all laid out in the following form: "Name/Reference". The first part (Name) has the character style "Product Name" while the second part (/Reference) has the character style "Product Ref". When we select the style "Product Name" in IndexMatic and we send the query /[a-z]+\/[0-9]+/i, the script returns no result. How can we make it work, and is it possible to render the items in the form: "Name (Reference): page numbers"?

[BOTH VERSIONS] IndexMatic cannot directly target mixed-style content (unless you select no style!), but there is a simple workaround: gather the required character styles in a style group. This allows you to target that group in the IndexMatic's Style panel—style groups are listed in the form: [group_name] *. Here is how to proceed with your example:

1. In InDesign, create a character style group, say "Product", and move the existing styles ("Product Name" and "Product Reference") in the newly created group.

2. Start IndexMatic. In the Character Style dropdown list, select: [Product] *.

3. Send the query: /([a-z]+)\/([0-9]+)/i => $1 ($2).

Inner Spaces and Whole Words

• I want to find matches that contain white spaces, like "sold out" or "back pay". Do I need to uncheck the “Whole Word” option?

[BOTH VERSIONS] No. The purpose of the Whole Word option is to ensure that a match is not preceded or followed by a character that belongs to the current Alphabet. This does not preclude the presence of spaces or non-alphabetic characters within the search key.

For example, with the default settings (Whole Word active) the query back pay finds any occurrence of "back pay", but ignores a string such as: "rollback pay".

It is generally useful to disable Whole Word when a partial token can be found in several expressions that unambiguously refer to the same topic, for example: modern/W => modernity

Assumed that the substring "modern" can only occur in words that relate to modernity, the key:

    modern/W

is much more efficient than a complete pattern like:

    /modern(i(sm|ty|ze))?/

Automatic Mode vs. Query modes

• I was trying to generate an index based on a character style and I found out that IndexMatic only considers single words, but not compund expressions like “The Vice President” even if the complete expression (including the spaces) has an appropriate character style applied. Do I have a possibility to grab complete runs of character-styled characters?

[BOTH VERSIONS] You had probably invoked the Automatic Mode, which is not appropriate for your goal. In Automatic Mode, IndexMatic only grabs words—in the sense of the current Alphabet. To get extended results, you have to select the Single Query (or Query List) mode in order to send your own command(s). Here are some usual queries when you are filtering content by character style:

To globally capture every character-style run, use:
    /.+/

To capture words-and/or-space strings only, use:
    /[\w ]+/

To capture words only, use:
    /\w+/

3/ Basic Queries

Term Rewriting / Subtopics / Cross-References

• I would like to index words that does not actually appear on the pages (e.g. indexing "France" on pages that only talk about "Paris"). How to do?

[BOTH VERSIONS] Take advantage of the rewriting-operator (=>). Every occurence of "Paris" in the document can be rendered "France" in the index, using the following query:

   Paris => France

Of course you can both generate the topics "Paris" and "France" by these two queries:

   Paris
   Paris => France

The first line above indexes "Paris" as itself, while the second line creates alongside the topic "France" (from the same matches).

Another approach is to output "Paris" as a subtopic of "France":

   Paris => France > $0

The above query is more relevant in that it allows to manage other France-related subtopics, for example:

   Bordeaux => France > $0

A compact way to write together the two queries is then:

   /Paris|Bordeaux/ => France > $0

And the resulting index looks like:

   France
      Bordeaux: page numbers
      Paris: page numbers

In addition, if you want to get "Paris" as a first-level heading in your index, you can easily redirect the reader to the topic "France" by adding the following cross-reference (note the double slash at the beginning of the query):

   // Paris => See France

Finally, our complete query list could be:

   // Bordeaux => See France
   // Paris => See France
   /Paris|Bordeaux/ => France > $0

• In a topic like "FRANCE" I would like to have a sub entry, e.g. "Paris", which does not contain any page number and only refers the reader to an external topic named "PARIS". How to?

[BOTH VERSIONS] The syntax for cross-references allows you to declare a "See..." reference from any existing (or non-existing) topic or subtopic. Simply format the “fake term”—see Manual, page 20—as a formal subtopic:

    // FRANCE > Paris => See PARIS.

Now consider a query list having the following lines:

    ...
    FRANCE
    PARIS
    // FRANCE > Paris => See PARIS.
    ...

The final index will look like:

    …
    FRANCE: page numbers
        Paris: See PARIS
    …
    PARIS: page numbers
    …

Dealing with Plural Forms

• Given a list of words in singular form, is it possible to capture both the singular and the plural forms?

[BOTH VERSIONS] IndexMatic is not able to deduce alone what is the plural form of a word, so it is necessary to specify the corresponding alternatives in the queries.

To index singular and plural forms—or other variants—under the same heading, the best way is to extend the original words to regular expressions. Here is the most common example, including the letter 's' at the end of the word:

    /cats?/ => cat

which can be also expressed:

    /(cat)s?/ => $1

So, when you have several items based on the same plural transformation, you can easily factorize the keys as follows:

    /(cat|dog|snake)s?/ => $1

Of course, you have to deal with a number of special cases:

    /stor(y|ies)/ => story
    /wom(a|e)n/ => woman
    /person|people/ => person
    etc.

Queries and White Spaces

• We want to capture the string " USD" (with a space before), but the query: " USD" seems to be interpreted as "USD" without space. Why?

[BOTH VERSIONS] When a key is based on a simple token (with no starting slash), spaces at the beginning of the string are automatically ignored. Similarly, trailing spaces are ignored if the key has no ending slash. Study the following queries:

    sample
        sample/w
      sample    => Words > $0

In each case the considered token is just "sample" (without space).

To compel IndexMatic to take ending/trailing space(s) into account, simply enclose the key between slash characters:

    / USD/

Note that the expression is then parsed as a regular expression, but this has no side-effect unless you use pattern-specific operators.

Website Names, URLs

• Is it possible to index every website mentioned in my document and to render the entries in the form:
"name [URL], page numbers"?

[BOTH VERSIONS] This primarily depends on how these elements are treated in the document.

If a specific character style is applied to website names and/or their URLs, there is no difficulty in retrieving these items using the Style filter and a generic query like: /.+/

If the data are not styled, you have to provide your own wordlist to grab website names (as the script cannot recognize a priori the name of a website in the text.)

If you only have to grab the URLs, send a query like:
    /(http:\/\/|www\.)[^ ]+/I
or a more sophisticated one. This approach can be used as a preparatory step to identify websites. You can then get the results from the Hit Report and build an improved query list that considers both websites' name and URL.

Key Length / Grouping Alternatives

• I found it very useful to target a list of subtopics with queries like:
/John|Bill|Kate/=>Authors>$0. But, how many terms can be in such a query? I have several hundreds of such terms. Can this number be handled by the query? Or what would be the alternative approach?

[BOTH VERSIONS] The maximum length of an IndexMatic key is set to 172 characters since version 2.025. This allows to write complex regular expressions, but indeed you cannot handle a hundred of alternative expressions this way. The solution is to use a regular Query List:

    John => Authors > $0
    Bill => Authors > $0
    Kate => Authors > $0
    ...

However, you still can optimize the list with a set of grouped alternative terms, provided that each key does not use more than 172 characters. For example:

    // A...
    /Aaron|Adelle|Alban|Andre|Arnold|Ava/ => Authors > $0
    // B...
    /Barton|Beatrice|Benjamin|Brandon|Breanna/ => Authors > $0
    // C...
    /Celeste|Charmaine|Chuck|Constance|Curt/ => Authors > $0
    ...

Using the "\w" Metacharacter

• What is the exact scope of the \w metacharacter?

[BOTH VERSIONS] The \w metacharacter automatically fits the current Alphabet settings: it matches any character of the selected alphabet, and optionally matches the hyphens, the digits, the apostrophe, and/or the underscore—depending on the checked boxes in the Alphabet panel. See the user's guide to have complete details on these features.

Note that the behavior of \w is IndexMatic-specific. In pure JavaScript regular expressions, \w only matches an alphanumeric character, including "_". Hence, if you want to make \w behaves as in JavaScript, select the ASCII alphabet and check the “Allow digits” and “Allow underscore” boxes. Otherwise, you can still use the character class [a-zA-Z0-9_].

Extracting XML Data

• Is IndexMatic able to parse text like:
"...<index>New Orleans</index>..."?
And is it possible to extract the attribute as well? Like:
"...<index entry="New Orleans, LA">New Orleans</index>..."

[BOTH VERSIONS] To grab the content of the <index> element, send the following query:
    /<index>([ \w]+)<\/index>/ => $1

And to grab the attribute:
    /<index entry="([ \w,]+)">/ => $1

Special Space Characters

• At a specific location of a regular expression, I want to specify "thin space" OR "hair space" rather than the IndexMatic's generic space. Is it possible?

[BOTH VERSIONS] Simply enter the explicit character class: [~<~|] (GREP syntax), or [\u2009\u200A] (Unicode code points). See also the user's guide, page 22.

Note. — Whatever the Generic Space settings, IndexMatic alway considers the special characters you specify.

4/ Advanced Queries

Punctuation

• What is the most generic way to indicate a punctuation character in a regular expression?

[BOTH VERSIONS] The Unicode metacharacter \p{P} will capture any punctuation sign. Some refinements are available (see the user's guide, page 22).

Redundant Matches

• In the book I'm indexing I have three variants of first names ("P. H. Nielsen", "L.-D. Nisipeanu", and "G. Kasparov") which I grab from character-style runs. My queries are the following:
    // 1. Catches "P. H. Nielsen" etc.
    /([A-Z]\. [A-Z]\. )([A-Z]\w+)/ => $2, $1
    // 2. Catches "L.-D. Nisipeanu" etc.
    /([A-Z]\.\-[A-Z]\. )([A-Z]\w+)/ => $2, $1
    // 3. Catches all like "G. Kasparov"
    /([A-Z]\. )([A-Z]\w+)/ => $2, $1
But Nielsen is then duplicated in the output as both "Nielsen, H." and "Nielsen, P. H." How to fix this?

[BOTH VERSIONS] Since IndexMatic does not support the “lookbehind” assertion, it is not possible to prevent "P. H. Nielsen" from being detected by your third query, which in this case is redundant since your first query already treats that string. This is a common issue when one need to control the very beginning of the input string. By chance you can solve the whole problem through a single query:

    // Catches all cases from a single pattern:
    /([A-Z]\.(?:[ -][A-Z]\.)?) ([A-Z]\w+)/ => $2, $1

This works because the ? operator in the middle of the query is greedy, which forces the engine to grab "P. H." rather than "P." alone. As a general strategy, when you need to capture variants of a single data type it's better to address all the cases from a unique query. Using multiple queries causes redundancy if two distinct regular expressions have a chance to capture the same string.

Note. — The syntax (?: in the above pattern declares a non-capturing parenthesis. This allows to create an optional group,
(?:[ -][A-Z]\.)?
without increasing the number of captured placeholders, so $2 always refers to the second part of the global pattern, ([A-Z]\w+).

Stats on letters

• I want to report all letters contained in a document, including non-Latin letters, with their related frequency. Can this be done through the “Hit Report” feature?

[PRO VERSION]

1. Enter the following (single) query:
    /[\p{Ll}\p{Lu}\p{Lt}]/IW

2. Set the Entry Case to [No change].

3. Press the Hits... button.

Note. — The \p metacharacter is not affected by the Alphabet settings, you can always use it to search characters from their Unicode properties.

Using alone the "$" Symbol

• We are testing different regular expressions in order to find the most relevant in URL extraction. Can IndexMatic display what pattern has produced each specific set of results?

[BOTH VERSIONS] In the query term, the $ symbol always represents the original key in its literal form. For example:

    /[a-z]{3}\d/ => $

will report in a single topic, "[a-z]{3}\d", every page that contains a sequence of three letters followed by a digit.

Hence, you can easily 'backup' the relationship between each pattern and the found terms. In your case, a possibility is to output the patterns as first-level topics and the URLs as related subtopics:

    /pattern1/ => $ > $0
    /pattern2/ => $ > $0
    etc.

5/ Output

Multiple indexes

• My problem is to create multiple indexes for a book (index of places separately from the index of personal names, etc.). Is IndexMatic capable of making such multiple indexes?

[BOTH VERSIONS] Basically, IndexMatic does not manage multiple indexes at the same time. You have to configure options/queries first to build the PLACES index, then you need to re-run the script and apply new settings in order to create the NAMES index, etc. Switching to specific prepared wordlists from the Query Editor should help you to quickly achieve these tasks.

If each index targets a one-level topic set, another option is to create a single query list based on subtopics, where each subtopic represents in fact a sub-index (NAMES, PLACES, etc.), for example:

    /Boston|Atlanta|Paris/ => PLACES > $0
    /John|Bob|Jiri/ => NAMES > $0

Although this results in a single file, your final entries will be rendered in the form:

    NAMES
        Bob 5, 12-13, 20...
        Jiri 14, 18, 22...
        John 17, 20-23...
    PLACES
        Atlanta 7, 9, 12-13...
        Boston 15...
        Paris 12-15, 17-22...

6/ Limitations and Known Issues

Non-Latin Alphabets

• IndexMatic doesn't work properly with Russian language. Have you any plan to extend it for support Cyrillic alphabet?

[BOTH VERSIONS] IndexMatic can handle any Unicode character from the Basic Multinlingual Plane but indeed it does not provide any practical way to specifically target and sort Cyrillic-based terms. So far, only Latin alphabets are fully supported in this regard. We are studying the possibility of adding non-Latin alphabets in a future release.

Freezed Progress Bar / Table indexing

• The progress bar seems to freeze when I'm indexing a long document including a number of tables.

[BOTH VERSIONS] Due to the heavy (and someway inefficient) structure of table components in the InDesign Document Object Model, indexing table contents can be particularly slow, which leads to long stage in the progress bar. Be patient!

Delayed result

• I thought IndexMatic had crashed, but after a long delay the final index suddenly appeared. How to know whether the script is still running?

[BOTH VERSIONS] Indeed, some Mac-platform users reported that the progress bar may disappear as the indexing process is ongoing. More likely the progress bar is then hidden by another element, but this is—of course!—an unexpected behavior. We are investigating on this issue.

“Indirect” Character Styles

• When indexing based on character styles, IndexMatic seems to find only those character styles that are explicitely applied and not those that are applied by "nested styles" or a "GREP style" within a paragraph style. Can you make the script find character styles that are formatted with a nested/GREP style?

[BOTH VERSIONS] IndexMatic indeed does not inspect indirect styles, it only considers formal Character and Paragraph styles, whatever the way they are enriched or overridden.

As suggested by Laurent Tournier, a workaround is to use a companion script in order to convert local formatting to pure character styles. Some interesting tools are reviewed in this post from InDesign Secrets: “Free Scripts Help Fix Word Formatting”.

Stop Words?

• Is there a way to prevent usual words like ‘the’, ‘is’, ‘are’ from being indexed in Automatic mode?

[BOTH VERSIONS] You can filter short worts by setting the “Min. Length” value to 4 or 5 in the Search Mode panel. However, IndexMatic does not provide stopword lists (as seen in Wordalizer). We may add this feature in a future release.