Implementing Autocomplete and More Like This using ASP.NET Core, Elasticsearch and NEST 5.x (Part 4/4)

Welcome to the final part of this series!

Finally, after some blood, sweat, and a few keystrokes, you’ve arrived at part 4!

In this part we’ll be implementing Autocomplete together with More like this functionalities!

You will be astonished to see how easy it is to implement these seemingly hard tasks using the NEST client together with ASP.NET Core.

Autocomplete

The implementation of our Autocomplete functionality will happen in the following steps.

1. Put together the fluent DSL expression that will return to us a search response.
2. Create an AutocompleteResult class that we’ll be using to return our data with.
3. Extract the actual data from the returned ISearchResponse object and map it to AutocompleteResult.
4. Expose an Autocomplete API endpoint from our RecipeController.

Step 1. The fluent DSL expression

First, let’s start by adding a method called Autocomplete in our SearchService class.

public async Task<List<AutocompleteResult>> Autocomplete(string query) { }

For this part, we’ll be making use of Elastic’s suggest query. The raw query that we’ll be aiming to achieve using fluent DSL looks like this:

{
 "suggest": {
   "recipe-name-completion": {
     "prefix": user-inputted query,
     "completion": {
       "field": "name",
       "fuzzy": {
         "fuzziness": "AUTO"
       }
     }
   }
 }
}

Nothing too unusual. I’ll explain the meaning of each field a bit later when we go through our fluent DSL expression line-by-line.

Here’s how it’s going to look:

var response = await this.client.SearchAsync<Recipe>(sr => sr
                .Suggest(scd => scd
                    .Completion("recipe-name-completion", cs => cs
                        .Prefix(query)
                        .Fuzzy(fsd => fsd
                            .Fuzziness(Fuzziness.Auto))
                        .Field(r => r.Name))));

The name that we pass as a first parameter for the Completion method will be used a bit later to extract the actual suggestions.

Let’s examine it line-by-line.

var response = await this.client.SearchAsync<Recipe>(sr => sr

Define a variable called response and invoke ElasticClient‘s SearchAsync.

.Suggest(scd => scd

We specify that what we’ll be sending is a query of type Suggest.

.Completion("recipe-name-completion", cs => cs

We specify that the Suggest query is actually going to be a CompletionSuggester and we give it a name recipe-name-completion.

.Prefix(query)

As a prefix field (see the raw query above) we pass in the user inputted query.

.Fuzzy(fsd => fsd
.Fuzziness(Fuzziness.Auto))

We specify that the query is going to be a Fuzzy one and we set the Fuzziness level as Auto. Auto changes dynamically depending on the length of the query. (see the docs)

.Field(r => r.Name))));

Finally, we tell the CompletionSuggesterDescriptor (one of these NEST objects that are responsible for “describing” our C# query to Elastic) that the field it should query is the name field of our Recipe class.

Remember when in part 2 we marked the Recipe class’ Name property with the [Completion] attribute?

[Completion]
  public string Name { get; set; }
  

If we hadn’t done that we wouldn’t have been able to query the Name field with a CompletionSuggester.

Step 2. The AutocompleteResult class

In your project’s Models folder, create a new class called AutocompleteResult with the following implementation:

public class AutocompleteResult
{
    public string Id { get; set; }

    public string Name { get; set; }
}

This will be the object that sits behind every suggestion. We need the Id so when the user actually clicks a recipe’s name we can send him straight to it.

Step 3. Extracting the suggestions from the ISearchResponse

In the Autocomplete method we created earlier, just below the fluent DSL expression we created, append the following line.

List<AutocompleteResult> suggestions = this.ExtractAutocompleteSuggestions(response);

Now press Ctrl + . and choose the Generate method option. Visual studio should generate for you the following method body:

private List<AutocompleteResult> ExtractAutocompleteSuggestions(ISearchResponse<Recipe> response) 
{ 
    throw new NotImplementedException(); 
}

Now, let’s create a list that will hold our AutocompleteResult objects.

List<AutocompleteResult> results = new List<AutocompleteResult>();

The suggestions will be contained in the response.Suggest dictionary under the name that we gave our Completion query earlier. (in our case – "recipe-name-completion").

Each suggestion contains a collection called Options which actually holds the suggested objects. (yeah, it’s a bit tricky). We’ll be using LINQ to select those.

var suggestions = response.Suggest["recipe-name-completion"].Select(s => s.Options);

Now what we need to do is traverse each returned IReadOnlyCollection<SuggestOption<Recipe>>, get the suggested Recipe object and map it to AutocompleteResult.

foreach (var suggestionsCollection in suggestions)
{
    foreach (var suggestion in suggestionsCollection)
    {
        var suggestedRecipe = suggestion.Source;

        var autocompleteResult = new AutocompleteResult
        {
            Id = suggestedRecipe.Id,
            Name = suggestedRecipe.Name
        };

        results.Add(autocompleteResult);
    }
}

Alternatively, we can do that in a more functional way using LINQ and AddRange.

suggestions.ToList().ForEach(s =>
{
    results.AddRange(s.Select(opt => new AutocompleteResult
    {
        Id = opt.Source.Id,
        Name = opt.Source.Name
    }));
});

// Or even

var results = matchingOptions
                .SelectMany(option => option
                .Select(opt => new AutocompleteResult() { Id = opt.Source.Id, Name = opt.Source.Name }))
                .ToList();

Now that we’ve extracted the results. We are ready to expose our API Autocomplete endpoint.

Step 4. Exposing the API endpoint

Open the RecipeController class and add the following method:

[HttpGet("autocomplete")]
public async Task<JsonResult> Autocomplete([FromQuery]string query)
{
    var result = await this.searchService.Autocomplete(query);
    return Json(result);
}

Now we’re able to access our Autocomplete functionality by making a GET request to appurl/api/recipe/autocomplete.

More like this

The MoreLikeThis implementation is going to be a bit simpler than the Autocomplete. It is going to happen in 2 steps:

1. Put together the fluent DSL expression.
2. Expose a MoreLikeThis API endpoint from our RecipeController.

Step 1. Putting together the fluent DSL expression

Let’s start by adding a MoreLikeThis method into our SearchService class.

public  async  Task<SearchResult<Recipe>> MoreLikeThis(string  id, int  page, int  pageSize)
{
}

The fluent DSL being just a strongly typed C# version of the raw JSON queries that Elastic requires, building it would imply that we know what is the actual raw query we expect.

Let’s take a look:

{
  "from": (page - 1) * pageSize,
  "size": pageSize,
  "query": {
    "more_like_this": {
      "fields": [
        "ingredients"
      ],
      "like": [
        {
          "_index": "recipes",
          "_type": "recipe",
          "_id": "id"
        }
      ]
    }
  }
}

Piece of cake.

We have the usual from and size fields used for the pagination and a query object which simply states that we’ll be comparing our recipes by their ingredients field + the index and type that elastic should be looking in.

This means that our DSL query will look like:

1. await client.SearchAsync<Recipe>(s => s
2.                .Query(q => q
3.                    .MoreLikeThis(qd => qd
4.                        .Like(l => l.Document(d => d.Id(id)))
5.                        .Fields(fd => fd.Fields(r => r.Ingredients))))
6.                        .From((page - 1) * pageSize)
7.                        .Size(pageSize));

You should be comfortable looking at these at this point. Anyhow, let’s go through it line by line.

await client.SearchAsync<Recipe>(s => s
               .Query(q => q

Invoke the client’s SearchAsync and start describing the query using NEST’s QueryContainerDescriptor.

.MoreLikeThis(qd => qd

Specify that the query we’ll be sending is a MoreLikeThis.

.Like(l => l.Document(d => d.Id(id)))

Specify that we’ll be querying for a document similar to the one that matches the id passed into the method using NEST’s LikeDocumentDescriptor.

.Fields(fd => fd.Fields(r => r.Ingredients))))

Specify that we want the documents to be compared by their ingredients field.

.From((page - 1) * pageSize)
.Size(pageSize));

Get documents based on the pagination info.

The implemented method now looks like this:

public async Task<SearchResult<Recipe>> MoreLikeThis(string id, int page, int pageSize)
{
    var response = await this.client.SearchAsync<Recipe>(s => s
        .Query(q => q
            .MoreLikeThis(qd => qd
                .Like(l => l.Document(d => d.Id(id)))
                .Fields(fd => fd.Fields(r => r.Ingredients))))
                .From((page - 1) * pageSize)
                .Size(pageSize));

    return new SearchResult<Recipe>
    {
        Total = response.Total,
        ElapsedMilliseconds = response.Took,
        Page = page,
        PageSize = pageSize,
        Results = response.Documents
    };
}

Now let’s make it available for our API users by exposing an endpoint for it.

Step 2. Exposing a MoreLikeThis endpoint

Open the RecipeController class and add the following method:

[HttpGet("morelikethis")]
public async Task<JsonResult> MoreLikeThis([FromQuery]string id, int page = 1, int pageSize = 10)
{
    var result = await this.searchService.MoreLikeThis(id, page, pageSize);
    return Json(result);
}

This will allow us to access the more like this functionality by making a GET request to appurl/api/recipe/morelikethis.

---

Congratulations!

You’ve made it until the end. You have now learned:

1. How to enable your .NET application to “talk” to your Elastic instance through NEST’s ElasticClient.
2. How to provide an ElasticClient instance throughout your application.
3. How to use NEST AttributeMapping.
4. How to use the ElasticClient object to index data into Elastic.
5. How to build an Elastic query using C# and fluent DSL.
6. How to make use of the Completion and MoreLikeThis queries.

Thank you for following this tutorial. If there’s anything that is still unclear to you, don’t hesitate to contact me or post a comment.

Now take your newly aquired knowledge and make some killer apps!