Knowledge Base

Terminalfour Search Implementation 

This course is aimed at front-end developers and takes you through the tasks that you will perform when working on implementing new/additional Terminalfour Search implementations.

What this training will cover:

  • Setting up a site for a crawl
  • Crawling and indexing a site
  • Creating a search results page (including filters and facets)
  • Tuning and customizing the order of the results

Prerequisites to completing the course

  • HTML, JavaScript and CSS familiarity

What you will need before you begin

Before you start, you'll need:

  • Access to Terminalfour Search Crawler
  • A SearchStax Site Search account
  • Access to Terminalfour training environment

Overview of Solution Components

Overview of Solution Components

Solution Components

Prepare your site for a crawl

Before a search engine can show your site in search results, it needs to explore and understand your content.

This section explains how to get your website ready for that process, including steps to make sure pages can be found, loaded, and understood correctly. By preparing your site for crawling, you help web crawlers index your content more accurately.

This section outlines the essential steps to prepare your website for crawling, including:

  • configuring server responses
  • managing URL accessibility, and
  • optimizing site and content structure for discoverability.

Good pre-crawl preparation ensures that the indexing process is efficient and comprehensive, reducing the risk of missed or misinterpreted content.

A website has been published for you to crawl, follow the instructions below to locate and view the site.

How to locate and explore your sample site

Create a Search App

  • A search App is the foundation of a Site Search project.
  • It serves as a container for your data, analytics, and search configurations.
  • It records the history of searches, clicks, searches per session and other analytics over a period.

How to create a search app

  1. Your trainer will invite you via an email from SearchStax to join the Terminalfour_ClientTraining account. Follow the instructions in the email to access the account.
  2. Go to https://searchstudio.searchstax.com/
  3. Log in with your Email Address and Password.

Sign-In Screen

  1. In the SearchStax Terminalfour_ClientTraining account, go to https://searchstudio.searchstax.com/search-apps

         SearchStudio Dashboard >> All Apps >> CREATE APP

All Apps - Create App

App Creation Screen

  1. Fill in the details
    1. App name: name your profile, your trainer might suggest a naming convention to follow. i.e. yourName
    2. Region: choose one close to your location
    3. Environment: Production
    4. Platform: Custom App
    5. Version: N/A
  2. Click CREATE
  3. Your Searchstax Search App queries the index of your site using API integrations.
    Make a note of your API endpoint and token, you will paste them into a Terminalfour Crawler Configuration later. 

          You will need:

    1. Searchstax EndPoint:

Search APIs  

      1. Edit the Endpoint to remove “emselect” from the end of the endpoint value e.g., https://searchcloud-1-us-east-1.searchstax.com/29847/trainfruituniversity-6003/
  2.  
    1. Read and Write Token: 

Tokens

      1. e.g., 65efa070b8d6258d4db2e1a2b2cb57edc09afdf4
        Note: To return to the API screen at any time, go to
        App Settings >> All APIs

Log in to Search Crawler

  • Terminalfour Search Crawler is used to crawl and build an index of your site.
  • A crawl creates an index of your site and connects to your Searchstax App using APIs.
  • Log into Terminalfour Search Crawler using the credentials provided by your trainer. 

How to log in to Terminalfour search crawler

  1. Open a web browser and navigate to the Terminalfour search crawler.

https://search.terminalfour.net/configuration-list/t4-training

Search Crawler Log In Screen

  1. Enter the Email and Password credentials provided by your trainer; then click Log in.
    1. Email: provided by your trainer.
    2. Password: provided by your trainer.
  2. Terminalfour Search opens.

Add a Configuration

Use configurations to guide and optimize how Terminalfour Search scans, indexes, and processes your content.

Add a new configuration

How to add a new configuration

  1. To add a new configuration, go to:

     Terminalfour search >> Home >> Add new configuration 

    1. General

              This section allows users to set configuration name, SearchStax credentials and choose the type of search crawl to configure.

      1. Configuration name: Provide a unique name for the configuration, e.g.: example-name. 
      2. Searchstax endpoint: API integration endpoint found in SearchStax settings.

Edit the Endpoint to remove “emselect” from the end of the Endpoint value.

Edit the endpoint

i.e., https://searchcloud-1-us-east-1.searchstax.com/29847/trainfruituniversity-6003/

      1. Searchstax Token: Enter 'Read & Write' token found under SearchStax settings.

e.g., 3b6063638995h9b61440c4016cb759be9c7d44f1

Searchstax Token

      1. Test connection: Click Test connection.

Test connection button

If you do not get a successful connection review your Searchstax endpoint and Searchstax token credentials.

      1. Crawl type: Choose crawl type -in this exercise we will choose a Web crawl. 
  2.  
    1. Origin URL and crawl settings

Define the base URL for a website. Specify the parameters and rules for how search engine should crawl and index the site's content.

      1. Seed: Enter the starting URL where the web crawler will begin exploring the site. 

i.e.: https://training.terminalfour.net/t4-fruit-university/
You can add multiple seed URL’s by clicking Add another if you want to add sub domains, or other sites.

      1. Success threshold: Minimum percentage of documents that must be gathered for an update to succeed.
      2. Page crawl delay: Number of seconds the crawler will delay between success requests to the server.
      3. Timeout: Max. time (in seconds) the crawler will wait for a network response before abandoning the request.
      4. Fetch interval: Default number of hours between re-fetches of a page. Set as 1 hour.
      5. Sitemap URL: This is a way to ensure all URL’s/pages in your site are found by the crawler, even if a link to a page is not found elsewhere on your website.
        1. Enter the domain where the XML version of your sitemap can be found. e.g.: https://docs.terminalfour.com/sitemap.xml
      6. URL filter regex: Enter matching pattern to determine whether URL is included or ignored by the crawl.

+^https://training\.terminalfour\.net/t4-fruit-university/

You MUST have Excludes listed before your Includes (i.e., - before +) AND you MUST escape special characters:

-^https://training\.terminalfour\.net/t4-fruit-university/athletics
+^https://training\.terminalfour\.net/t4-fruit-university/

    1. Content

Define the type of content that can be processed by search engine along with the maximum size for each content item.

      1. Index start comment: A phrase to indicate where extraction should start. E.g.: Search text crawl start 
      2. Index end comment: A phrase to indicate where extraction should end. E.g.: Search text crawl end 
      3. Content size limit: Size limit in kilobytes of the content to be downloaded for each page.
      4. Crawl documents: Include pdfs, docx, pptx, xlsx, odt, ods, odp.
  2.  
    1. Mappings

Mappings are values used in search queries to refine results based on specific metadata properties, which are linked to crawled properties for content indexing.

      1. Static values: A constant value that is added to the specified field on every item that is indexed.

We will leave this unpopulated for now and return to it later in the training.

      1. Dynamic values: For each item indexed, take the value from a specified meta tag on the page and assign it to the selected field in your index.

We will leave this unpopulated for now and return to it later in the training.

  1. Click Save

You will now return to the Available Configurations screen.

Run a Crawl

Now that you have saved your Configuration, you are ready to run your first crawl.

Run a crawl

How to run your first crawl

  1. In Terminalfour Search, go to Home >> Configurations.

Locate your Configuration (you can use the filter box provided).

  1. Click Actions >> Run crawl.

After a few minutes your crawl should be complete.

Check Results of a Crawl

After a few minutes, the initial crawl of your website will be complete. 

Once finished, review the results of your crawl. You have several options:

  1. View crawl stats in Terminalfour Search
    • Use this to confirm that your Crawl has created an index of your URLs
  2. Preview your search engine results using Searchstax’s Preview
    • Use this to confirm that your index has integrated with the Search App
  3. View your Search Engine Results Page (SERP) using Hosted Search Experience in Searchstax
    • Use this to see the starting point of your Search Engine Results Page (SERP)

View Crawl Stats

A quick way to see what the crawler has indexed from your site is to click Crawl stats and:

  • Observe the number of URLs fetched
  • View fetched URLs
  • Check for any errors 

View crawl stats

How to View Crawls Stats

  1. In your Terminalfour Search, go to Home >> Configurations
    1. Locate your configuration and Click Crawl stats
      1. Observe the number of Total URLs (it should be 34) 
      2. Click View fetched URLs to observe the list of URLs found
      3. Click View parse errors URLs to observe any errors

Preview of search results

Now let’s confirm that your Terminalfour Crawler Configuration has successfully integrated with your Searchstax Search App.

  • Go to your Searchstax Search Application and use the Preview method.
  • The number of results should match the number of URLs fetched by your Terminalfour Crawler.

Search Preview

How to Preview

  1. In your Searchstax Search App, click Preview
    1. The Preview screen opens
    2. Check that all URLs are showing as results

Hosted Search Experience

The Hosted Search Experience is a prebuilt search interface hosted by SearchStax. It uses your Terminalfour Crawler’s index and the Searchstax Search App’s settings.

Hosted Search Experience

Hosted Search Experience View

How to open and share your Hosted Search URL 

  1. In your Searchstax Search App,
    Go to: Configurations >> Search UI >> Hosted Search Experience
    1. Click View to open the hosted page
    2. Enter a search query using *
    3. Copy the URL and share it in the training’s Zoom chat
    4. Observe that all your URLs are listed as results but not displaying any Titles or Descriptions 

Schedule a Crawl

Now that you have confirmed that the crawl has been configured correctly you can set a crawl schedule for future re-crawls of your site.

Configure a crawl schedule.

How to schedule a crawl

Schedule a crawl

  1. In your Terminalfour Crawler go to Actions >> Schedule Crawl
    1. Rate: Number of hours between crawls
    2. Start date: Choose a start date for the crawl
    3. Start time: Choose a start time for the crawl
    4. Timezone: Choose a time zone for the crawl

Result Fields

The Results Fields screen lets you configure two important features:

  • It lets you choose which data fields Solr will return in your search results.
  • It also lets you assign returned field values to specific display areas on the search-results “card.”

Result Fields

How to configure Result Fields

  1. In your SearchStudio application, go to Configurations >> Basic Configurations >> Results Fields
    1. Return field: select a field to map to the default Result Card.
    2. Label: optional label for your chosen field – useful if the original field name is not descriptive enough.
    3. Maps to Results Card Field: allows the SearchJS to map your chosen field to a specified Result Card Field.
  2. Delete the existing mappings of Title and Description and experiment by mapping others.
Return Field Maps to Result Card Field What it Does
url Url Determines where the result card goes when clicked
url Paths Displays the URL in the search result card
H1 Title  
content Description Displays the content field as the result item’s Description in the search
result card

If you don’t see any useful Search Fields to map you can update your crawler configuration to include some new mappings and come back here to complete your result fields configuration.

Mappings

Mappings are key to successful search implementation.

Review your website’s source code and identify metadata that would be useful to map into your search index.

Once you have mapped metadata into your search index you can utilize these as “Search Fields” to determine:

  • Search result ranking relevancy
  • Search result display cards
  • Search Result Facets and Filters

Mappings

How to Add Mappings

  1. In your Terminalfour Search Configuration
    1. Scroll down to Mappings.
    2. Add Static and Dynamic Mappings values – use the screen shots and tables below as a guide.

Static values

Static values

Static Field Name Field Type Field Value
siteName General Fruit

Dynamic values

Dynamic values

Dynamic Field Name Field Type Metatag Name
sectionType String sectionType
dc:title Text English dc:title
og:title Text English og:title
og:description Text English og:description
og:image String og:image
og:image:alt Text English og:image:alt
    1. Click Update.
      You will return to the Configurations listing screen.
    2. Click Actions >> Run crawl.
    3. After the crawl has completed, go to your Searchstax Search Application to observe the mappings.

Updating Result Fields

Once your updated Metadata Mappings have been added to your Crawler’s index it is time to utilize them in your Result Fields.

Go to your Searchstax Search Application and map your newly mapped fields to your Result Card.

Updating result fields

After adding the above fields click Publish.

Reload your Hosted Search Experience

Now that you have mapped fields to your Results Card, go to your hosted Search Experience to observe the changes to your displayed results.

Reload your hosted search experience

Promotions

Promotions in the Search Optimizations area help you feature key content at the top of search results. Use them to promote content for seasonal events, high-impact keywords, or specific campaigns.

Promotions are a Profile-level feature. Changes to the list of promotions can be assigned to one or more Profiles as required.

A Search Profile applies Data Filters first, then Ranking, Rules, Smart Ranking, and finally Promotions.

How to create a promotion

  1. In your SearchStudio, go to optimizations >> Promotions
  2. Enable this promotion: Ensure this is enabled.
  3. Click CREATE A PROMOTION.
  4. Give your promotion a name i.e., Bananas Foster and click CREATE.
  5. Search Term: Bananas.
  6. Condition: Contains.
  7. Click on the INDEXED ITEMS tab: 
  8. Search for Bananas and observe the results and their order
  9. Click on the Bananas Foster result to add this to the PROMOTED ITEMS area
  10. Click PUBLISH.
  11. Refresh your Hosted Search Experience and search for Bananas. observe that Bananas Foster is now the first result

A query for Bananas showing Bananas Foster as the Promoted result

When using the Hosted Search Experience and Preview, Promotions are easily identified by the Star icon to the right hand side of the promoted result(s).

Faceting

A facet is a type of dynamic filter designed to help users get where they need to quickly.

“Facets” are the lists of field values that appear on your application’s search page. The lists are populated from the set of matching items and are used to “drill down” and better “focus” the search.

Choose which fields will be used as facets in your search app. Once enabled, you will need to define at least one facet.

Faceting

How to configure Facets

  1. In your SearchStudio application, go to Configurations >> Basic Configurations >> Faceting
    1. Enable Faceting: Check the box to enable
    2. Facet Field: Select a field to use
    3. Label: This will appear on your SERP as the Heading/Label of the Facet
    4. Max. Facet Values: how many categories should appear in the Facet?
    5. Value Sorting: How to sort the categories in your facet
    6. Click + to add the facet field
    7. Click Publish

Search Fields

Your Search App matches user queries against a list of Search Fields. This list is a powerful tool for focusing the search on relevant material.

Search Fields

Click on an available field name to move it into the list of fields to search. The (X) button on the right removes a field from the active list.

Note the Reload Schema icon above the list of fields. If you can’t find a field in the list, try reloading the schema. Changes to your website’s Solr schema are not automatically updated in Site Search.

Available Fields

Stemming

Stemming is a process of removing and replacing word suffixes to arrive at a common root form of the word.

Stemming is one of several text normalization techniques that converts raw text data into a readable format for natural language processing tasks.

Stemming can be a very useful tool to help optimize your search results.

On your Fruit University hosted Search Experience:

  • A search for “blueberries” will yield a matching result.
  • A search for “blueberry” will not yield results.

A demonstration of stemming can be found here: https://text-processing.com/demo/stem/

  • Blueberries Stemmed to blueberri
  • Blueberry Stemmed to blueberri

Set your Search Fields to use a “stemming” field type i.e. a field set in your crawler configuration with a Field Type of Text English.
A field that is configured with a Field Type of Text English is identified with a suffix of _txt_en.

Using og_title_txt_en as one of your Search Fields for example, will yield results when searching for either blueberry or blueberries.

Stemming

How to use stemming in your Search Fields 

  1. In your SearchStudio application, go to Configurations >> Basic Configurations >> Search Fields
    1. Click Reload Schema: this will update the Available Fields to include fields recently added to your crawler’s
      configuration.
    2. Search Available Fields: use the search to find and select a field. Click a field to add it to the Search Fields on
      the right i.e.
      1. og_title_txt_en
      2. og_description_txt_en
      3. content
    3. Click Publish

How to use stemming

Spell Check

When a query returns no results, the Spell Check feature locates similar-looking words in the index (Data Driven) or in a custom list (Dictionary Driven). The Spell Check feature automatically runs a new query based on the similar word.

Spell Check is triggered by No ResultsIf Spell Check is not enabled, searching for "calender" will yield No Results

Enable the Spell Check to resolve the problem. 

Spell Check displaying no results for calender

How to configure Spell Check

  1. In your SearchStudio application, go to Configurations >> Basic Configurations >> Spell Check
  2. Enable Spell Checking: check this box
  3. Spell Check Method: Select Data-Driven.
  4. Search Field: Select text__spellcheck.
  5. Click Publish

Spell Check configuration enabled using a data-driven method with search field set to text__spellcheck

Search again for 'calender' and this time the Spell Check will yield a result

Spell Check enabled showing a result for calendar after a query of calender

Experiment with different Search Fields to determine which works best for your content.

Stopwords

Stop words are common words (like articles, prepositions, conjunctions, or pronouns) that do not add meaning to a search. Since they appear on almost every page, including them can skew the results. They’re useful in writing for sentence structure, but not in searches.

In your Hosted Search Experience, search for “bananas and blueberries”.

Observe that there are 36 results.

This is the total number of results that contain the word “bananas” OR “and” OR “blueberries”.

Many of your pages contain the word “and” but do not have any relevance to bananas or blueberries.

Add “and” to your list of Stopwords and re-run your search for “bananas and blueberries”.

This time, “and” is ignored by the query processor and you will get results that contain the word “bananas” OR “blueberries”.

How to configure Stopwords 

  1. In your SearchStudio application, go to Configurations >> Basic Configurations >> Stopwords
    1. Add Words: enter the word you want to add to the list of Stop Words, i.e. “and”.
    2. +: Click the ‘+’ symbol to add the word to the Stop Words dictionary
    3. Click Publish

How to configure stopwords

Auto-Suggest

The SearchStax Auto-Suggest feature helps website visitors by suggesting search terms based on an index of past user queries.

How Auto-Suggest Works

The dictionary is built nightly by looking at the last three months of search terms where the ClickThrough-Rate is greater than 10%.

Auto-Suggest

More detailed information available at https://www.searchstax.com/docs/searchstudio/searchstax-studio-auto-suggest/

How to configure Auto-Suggest

      1. In your Searchstax Search Application, go to
        Optimizations >> Auto-Suggest
      2. Enable Auto-Suggest.
      3. Suggestion Method:
        1. Data allows suggestions based on user search behavior.
        2. Dictionary uses the list of words you add to the Auto-Suggest configuration.
      4. Display: set the maximum of suggestions to display. I.e. 4.
      5. Display Suggestions After: set the minimum number of characters a visitor must type before suggestions begin to appear. I.e. 2.
      6. Add Word: add “Banana”.
      7. Click Publish.
      8. Preview your search and test your Auto-Suggest configuration by typing “ba”.
        Observe that “banana” appears as a search suggestion.
      9. Click on the “banana” suggestion and observe that a search is processed with the query of “banana”.
              Trouble shooting information for Auto-Suggest can be found at:

      https://www.searchstax.com/docs/hc/auto-suggest-issues/

      Data Filters

      Data Filters let you add both Include and Exclude filters to place limits on your search results.

      Experiment with Data Filters to include and/or exclude something from your search results.

      In the example below we will use Excludes to stop Fruit University’s Staff Directory items appearing in the search results.

      Exclude Result Matching

      How to Exclude Items from Search Results

      1. In your SearchStudio, go to Configurations >> Advanced Configurations >> Data Filters.
      2. Enable Data Filters: Ensure this is enabled.
      3. Exclude Result matching:
        1. SEARCH FIELD: sectionType_s.
        2. VALUE: People (note that values are case sensitive).
      4. Click Publish.
      5. Refresh your Hosted Search Experience and observe that “People” items are no longer appearing in your search results.

      Create a custom SERP Template

      To manage a template, we will use Terminalfour.

      Create a section in your Terminalfour instance using the credentials provided by your trainer. 

      How to log into Terminalfour

      1. Go to https://training.terminalfour.net/terminalfour/
        1. Username: S201 Implementer
        2. Password: S201 Implementer

      Hosting your SERP in Terminalfour

      The Hosted Search Experience is preconfigured to provide an appropriate search experience to the Search App’s index and settings.

      You can use the Searchstax Hosted Search Experience source code as the framework of your SERP template and customize it as required.

      How to generate a SERP Template

      Each Site Search App has its own Hosted Search Experience. To access it, navigate to: 

      Your Search App > Configurations > Search UI > Hosted Search Experience.

      How to generate a SERP template

      Click the View button to open the Hosted Search App in a browser window. You can copy the URL to share with stakeholders and coworkers.

      1. In your SearchStudio, go to Configurations >> Search UI >> Hosted Search Experience.
      2. Click VIEW, a new tab opens.
      3. View the pages Source Code.
      4. Copy everything and paste into your T4 Content Type Layout.

      Resources for Devs

      Template CSS and Styling

      There is a CSS file that can be used for default styling. It can be found within the SearchStax UI Source. The CSS file is linked to in the <header> there is also a <style> tag in the header of the source code. It is recommended to remove this and reference it from the Terminalfour Media Library.

      How to remove <style> from your template

      1. Go to your SERP content layout.
      2. Find <style lang="css"> (line 14).
      3. Delete all the style tag (lines 14 through to ~200).
      4. Replace with a T4 Media Tag pointing to the style file id:
        1. Click </>Generate T4 Tag.
        2. Select the Media tab.
        3. Click Select media.
        4. Locate the CSS file (go to Media Category: S201 Training Website).
        5. Click on the file: S201 SERP style.
        6. Select the path/* formatter.
        7. Click Copy to clipboard.

      <t4 type="media" id="12053" formatter="path/*" />

        1. The T4 tag will output the path to the CSS file. Update the code as follows: 

      <link rel="stylesheet" href="<t4 type="media" id="12053" formatter="path/*" />">

        1. Click Save changes.
        2. Preview your SERP to test the output. View the page Source to observe the <style> CSS is now replaced by a <link rel="stylesheet" href=""> instead of 184 lines of CSS code.

      Page source

      Search Profiles

      The SearchStax Site Search solution’s Search Profile Selector lets you choose which Site Search App and which Search Profile to view and modify. Search Profiles are customizable configurations that define a website’s search experience. They allow tailored configurations such as search fields and promotions, letting marketers fine-tune search experiences for different audiences. 

      Create two profiles

      1. Fruit
        1. Entire Fruit University search
      2. Directory
        1. Only Staff Directory search

      Search profiles

      Note that the Hosted Search Experience is configured for a specific search profile (or “model”), as shown by

      thesearchstax[model]=modelName

      parameter at the end of the URL. This provides a convenient way for developers to test competing models side-by-side.

      Using Hooks to select a profile

      Your Search Template is currently using your Main Profile (default).
      Use Hooks in your template to specify which profile to use.

      In the example below, we are changing updating the template to use the directory profile.

      Using hooks to select a profile

      Replace with: 

      const modifiedSearch = { ...searchObj };
      const searchstaxQuery = modifiedSearch.query;
      modifiedSearch.additionalProps = [{
                key: 'model',
                value: 'directory'
      }];
      return modifiedSearch;      Using hooks to select a profile (replace with code)

                                         After                                                                    Before

      After and Before

      Using Hooks to manipulate a searchField

      You have mapped dc:title to your result card’s Title.
      For your web pages, includes the site name i.e., Home | Terminalfour Fruit University.

      Using hooks to manipulate a searchField

      How to replace the suffix from your Title

      BEFORE

      How to replace the suffix from your title (Before)

      Add to Hooks, before the “return results”, at approximately line 156:

      results.forEach(result => {
        if (result.title) {
          result.title = result.title.replace(' | T4 Fruit University', '');
        }
      });

      AFTER

      How to replace the suffix from your title (After)

      Add Search to your Website’s Search Bar

      Now your search is ready to be added to your website.

      How to add the search solution to your website

      1. In your Page Layout Footer:
      2. Add the following code right before the </body>:

      <script>
      //Search Box
      document.addEventListener('DOMContentLoaded', function () {
       //must add input and button elements here. Make sure these exist on the page.
       const input = document.getElementById('customSearchInput');
       const button = document.getElementById('customSearchButton'); 

       button.addEventListener('click', function () {
         const query = input.value.trim();
         if (!query) return;
           const url = `https://training.terminalfour.net/t4-fruit-university/search/serpliam/?searchstax[query]=${encodeURIComponent(query)}&searchstax[page]=1`;
         window.location.href = url;
         });
         // Support pressing "Enter" inside the input
         input.addEventListener('keydown', function (e) {
         if (e.key === 'Enter') {
           e.preventDefault();
           button.click();
         }
       });
      });
      </script>

      1. Replace

      <input type="text" placeholder="Search" />   with

      <input id="customSearchInput" type="text" placeholder="Search" />

      1. Replace

      <div class="small-2 columns"><a href="#" class="button expand"><i class="fa fa-search"></i></a></div>

          with<div class="small-2 columns"><button id="customSearchButton" class="button expand"><i class="fa fa-search"></i></button></div>

      Feedback survey

      Congratulations on completing the Terminalfour Search Implementation training course.

      Please provide feedback in our on-line survey.