Welcome to the final part of this series!
Finally, after some blood, sweat, and a few keystrokes, you’ve arrived at part 4!
In this part we’ll be implementing Autocomplete
together with More like this
functionalities!
You will be astonished to see how easy it is to implement these seemingly hard tasks using the NEST client together with ASP.NET Core.
Autocomplete
The implementation of our Autocomplete
functionality will happen in the following steps.
- Put together the
fluent DSL
expression that will return to us a search response. - Create an
AutocompleteResult
class that we’ll be using to return our data with. - Extract the actual data from the returned
ISearchResponse
object and map it toAutocompleteResult
. - Expose an
Autocomplete
API endpoint from ourRecipeController
.
Step 1. The fluent DSL
expression
First, let’s start by adding a method called Autocomplete
in our SearchService
class.
public async Task<List<AutocompleteResult>> Autocomplete(string query) { }
For this part, we’ll be making use of Elastic’s suggest query. The raw query that we’ll be aiming to achieve using fluent DSL looks like this:
{
"suggest": {
"recipe-name-completion": {
"prefix": user-inputted query,
"completion": {
"field": "name",
"fuzzy": {
"fuzziness": "AUTO"
}
}
}
}
}
Nothing too unusual. I’ll explain the meaning of each field a bit later when we go through our fluent DSL expression line-by-line.
Here’s how it’s going to look:
var response = await this.client.SearchAsync<Recipe>(sr => sr
.Suggest(scd => scd
.Completion("recipe-name-completion", cs => cs
.Prefix(query)
.Fuzzy(fsd => fsd
.Fuzziness(Fuzziness.Auto))
.Field(r => r.Name))));
The name that we pass as a first parameter for the
Completion
method will be used a bit later to extract the actual suggestions.
Let’s examine it line-by-line.
var response = await this.client.SearchAsync<Recipe>(sr => sr
Define a variable called response
and invoke ElasticClient
‘s SearchAsync
.
.Suggest(scd => scd
We specify that what we’ll be sending is a query of type Suggest
.
.Completion("recipe-name-completion", cs => cs
We specify that the Suggest
query is actually going to be a CompletionSuggester
and we give it a name recipe-name-completion
.
.Prefix(query)
As a prefix
field (see the raw query above) we pass in the user inputted query.
.Fuzzy(fsd => fsd
.Fuzziness(Fuzziness.Auto))
We specify that the query is going to be a Fuzzy
one and we set the Fuzziness
level as Auto
. Auto
changes dynamically depending on the length of the query. (see the docs)
.Field(r => r.Name))));
Finally, we tell the CompletionSuggesterDescriptor
(one of these NEST
objects that are responsible for “describing” our C# query to Elastic) that the field it should query is the name
field of our Recipe
class.
Remember when in part 2 we marked the
Recipe
class’Name
property with the[Completion]
attribute?[Completion] public string Name { get; set; }
If we hadn’t done that we wouldn’t have been able to query the
Name
field with aCompletionSuggester
.
Step 2. The AutocompleteResult
class
In your project’s Models folder, create a new class called AutocompleteResult
with the following implementation:
public class AutocompleteResult
{
public string Id { get; set; }
public string Name { get; set; }
}
This will be the object that sits behind every suggestion. We need the Id so when the user actually clicks a recipe’s name we can send him straight to it.
Step 3. Extracting the suggestions from the ISearchResponse
In the Autocomplete
method we created earlier, just below the fluent DSL
expression we created, append the following line.
List<AutocompleteResult> suggestions = this.ExtractAutocompleteSuggestions(response);
Now press Ctrl + .
and choose the Generate method
option. Visual studio should generate for you the following method body:
private List<AutocompleteResult> ExtractAutocompleteSuggestions(ISearchResponse<Recipe> response)
{
throw new NotImplementedException();
}
Now, let’s create a list that will hold our AutocompleteResult
objects.
List<AutocompleteResult> results = new List<AutocompleteResult>();
The suggestions will be contained in the response.Suggest
dictionary under the name that we gave our Completion
query earlier. (in our case – "recipe-name-completion"
).
Each suggestion contains a collection called Options
which actually holds the suggested objects. (yeah, it’s a bit tricky). We’ll be using LINQ
to select those.
var suggestions = response.Suggest["recipe-name-completion"].Select(s => s.Options);
Now what we need to do is traverse each returned IReadOnlyCollection<SuggestOption<Recipe>>
, get the suggested Recipe
object and map it to AutocompleteResult
.
foreach (var suggestionsCollection in suggestions)
{
foreach (var suggestion in suggestionsCollection)
{
var suggestedRecipe = suggestion.Source;
var autocompleteResult = new AutocompleteResult
{
Id = suggestedRecipe.Id,
Name = suggestedRecipe.Name
};
results.Add(autocompleteResult);
}
}
Alternatively, we can do that in a more functional way using LINQ
and AddRange
.
suggestions.ToList().ForEach(s =>
{
results.AddRange(s.Select(opt => new AutocompleteResult
{
Id = opt.Source.Id,
Name = opt.Source.Name
}));
});
// Or even
var results = matchingOptions
.SelectMany(option => option
.Select(opt => new AutocompleteResult() { Id = opt.Source.Id, Name = opt.Source.Name }))
.ToList();
Now that we’ve extracted the results. We are ready to expose our API Autocomplete
endpoint.
Step 4. Exposing the API endpoint
Open the RecipeController
class and add the following method:
[HttpGet("autocomplete")]
public async Task<JsonResult> Autocomplete([FromQuery]string query)
{
var result = await this.searchService.Autocomplete(query);
return Json(result);
}
Now we’re able to access our Autocomplete
functionality by making a GET request to appurl/api/recipe/autocomplete
.
More like this
The MoreLikeThis implementation is going to be a bit simpler than the Autocomplete. It is going to happen in 2 steps:
- Put together the
fluent DSL
expression. - Expose a
MoreLikeThis
API endpoint from ourRecipeController
.
Step 1. Putting together the fluent DSL expression
Let’s start by adding a MoreLikeThis
method into our SearchService
class.
public async Task<SearchResult<Recipe>> MoreLikeThis(string id, int page, int pageSize)
{
}
The fluent DSL being just a strongly typed C# version of the raw JSON queries that Elastic requires, building it would imply that we know what is the actual raw query we expect.
Let’s take a look:
{
"from": (page - 1) * pageSize,
"size": pageSize,
"query": {
"more_like_this": {
"fields": [
"ingredients"
],
"like": [
{
"_index": "recipes",
"_type": "recipe",
"_id": "id"
}
]
}
}
}
Piece of cake.
We have the usual from
and size
fields used for the pagination and a query object which simply states that we’ll be comparing our recipes by their ingredients
field + the index and type that elastic should be looking in.
This means that our DSL query will look like:
1. await client.SearchAsync<Recipe>(s => s
2. .Query(q => q
3. .MoreLikeThis(qd => qd
4. .Like(l => l.Document(d => d.Id(id)))
5. .Fields(fd => fd.Fields(r => r.Ingredients))))
6. .From((page - 1) * pageSize)
7. .Size(pageSize));
You should be comfortable looking at these at this point. Anyhow, let’s go through it line by line.
await client.SearchAsync<Recipe>(s => s
.Query(q => q
Invoke the client’s SearchAsync
and start describing the query using NEST’s QueryContainerDescriptor
.
.MoreLikeThis(qd => qd
Specify that the query we’ll be sending is a MoreLikeThis.
.Like(l => l.Document(d => d.Id(id)))
Specify that we’ll be querying for a document similar to the one that matches the id
passed into the method using NEST’s LikeDocumentDescriptor
.
.Fields(fd => fd.Fields(r => r.Ingredients))))
Specify that we want the documents to be compared by their ingredients
field.
.From((page - 1) * pageSize)
.Size(pageSize));
Get documents based on the pagination info.
The implemented method now looks like this:
public async Task<SearchResult<Recipe>> MoreLikeThis(string id, int page, int pageSize)
{
var response = await this.client.SearchAsync<Recipe>(s => s
.Query(q => q
.MoreLikeThis(qd => qd
.Like(l => l.Document(d => d.Id(id)))
.Fields(fd => fd.Fields(r => r.Ingredients))))
.From((page - 1) * pageSize)
.Size(pageSize));
return new SearchResult<Recipe>
{
Total = response.Total,
ElapsedMilliseconds = response.Took,
Page = page,
PageSize = pageSize,
Results = response.Documents
};
}
Now let’s make it available for our API users by exposing an endpoint for it.
Step 2. Exposing a MoreLikeThis endpoint
Open the RecipeController
class and add the following method:
[HttpGet("morelikethis")]
public async Task<JsonResult> MoreLikeThis([FromQuery]string id, int page = 1, int pageSize = 10)
{
var result = await this.searchService.MoreLikeThis(id, page, pageSize);
return Json(result);
}
This will allow us to access the more like this functionality by making a GET request to appurl/api/recipe/morelikethis
.
Congratulations!
You’ve made it until the end. You have now learned:
- How to enable your .NET application to “talk” to your Elastic instance through NEST’s
ElasticClient
. - How to provide an
ElasticClient
instance throughout your application. - How to use NEST
AttributeMapping
. - How to use the
ElasticClient
object to index data into Elastic. - How to build an Elastic query using C# and
fluent DSL
. - How to make use of the
Completion
andMoreLikeThis
queries.
Thank you for following this tutorial. If there’s anything that is still unclear to you, don’t hesitate to contact me or post a comment.
Now take your newly aquired knowledge and make some killer apps!
Hi,Thanks for your good job.
I have one request.
Please share full source code of this course, or send to my email, if possible.
please source code in asp.net core 2.2.
Thanks a lot.
[Best Regard]
Hi dear.
I have one question.
I have a online store site with many categories contain variant products.
how to best solution for this matter, if i want to query between two tables and then create elastic search index by query result.
what’s the correct way for this issue?
The other hand, I have a complex query of some tables(sql server tables),And i want create index for this query.
At last, for example, when i search hardware, then read all products that are in Hardware categories.
Thanks a lot.
Hi.
Thanks for your nice article.
I have a one problem for implementing autocomplete suggester in asp.net core and sql server.
How to create index of multiple fields of some tables?
For example, I search by Product name or category name or product attributes and etc.
Thanks.
Can you give an example of what exactly you’re trying to accomplish?
Very good start for beginner. It works great with ES 6.7 but is not working with ES 7. Can you keep it up to date and update it to Elastic search 7 ?
Thank you! Once I have some free time left on my hands I’ll update it to ES7.
Can you please share the final code?
The final code (along with a client) is available on GitHub – https://github.com/dnikolovv/elasticsearch-recipes-nest-angular
hi
im using web api 2.2
and error in line
var response = await this.client.IndexManyAsync(mappedCollection.Skip(i * batchSize).Take(batchSize), index);
Error :
{Invalid NEST response built from a unsuccessful low level call on POST: /products/_bulk?pretty=true&error_trace=true}
OriginalException :
Request failed to execute. Call: Status code 404 from: POST /products/_bulk?pretty=true&error_trace=true
My web api project non angular
please help me
Hi there,
I’m not sure what the cause could be, as I can’t see the code. Check out the following links to see if you can find a solution there:
https://stackoverflow.com/questions/56076737/nest-aggregation-returns-error-invalid-nest-response-built-from-a-unsuccessful
https://stackoverflow.com/questions/54146686/invalid-nest-response-built-from-a-unsuccessful-low-level-call-on-post
I just define the model as below code
client.Indices.Create(“demo”, c => c
.Index()
.Map(c => c
.AutoMap()
.Properties(p => p
.Completion(c => c
.Name(n => n.Suggest))
.Keyword(k => k.Name(x => x.CompanyName))
.Nested(e => e
.Name(nm => nm.Employees)
.AutoMap()
.Properties(pps => pps
.Completion(c => c
.Name(n => n.Suggest))
.Nested(h => h
.Name(n => n.Houses)
.AutoMap()
)
)
)
)
)
);
In the result on ElasticSearch I see the field is “Suggest”: {
….
}
I define the search (autocomplete) for that.
var response = await ElasticSearch.Instance._elasticClient.SearchAsync(q => q
.Index(“demo”)
.Source(so => so
.Includes(f => f
.Field(f => f.CompanyName)
.Field(f => f.Id)))
.Suggest(su => su
.Completion(“suggestion”, cs => cs
.Field(f => f.Suggest)
.Prefix(input.CompanyName)
.Fuzzy(f => f
.Fuzziness(Fuzziness.Auto))
.Size(10)
)
)
);
I got the errors when it’s runing
Can’t found the field “Suggest”
I dont understand where is it wrong?
This one is json file
“id” : “1b3b6529-2abb-4165-b2e5-1afe8efd7843”,
“companyname” : “Company Name A”,
“employees” : [
{
“id” : “e2098915-fcdc-4e66-86b2-f94be9797747”,
“lastname” : “A”,
“firstname” : “AAAA”,
“salary” : 1000,
“birthday” : “2019-11-08T03:55:16.6224809Z”,
“ismanager” : true,
“hours” : 141166224815,
“houses” : [
{
“id” : “590e8f08-4273-40cd-b283-2802a1f234ae”,
“price” : 1000,
“address” : “123abc”,
“suggest” : [ ]
}
],
“suggest” : [
{
“input” : [
“AAAA”,
“bb”
]
},
{
“input” : [
“KK”,
“bb”,
“c”,
“s”
]
}
]
}
],
“suggest” : [
{
“input” : [
“Company”,
“Name”,
“A”
]
}
]
}
},
that above code is resolved but I can’t suggestion from sub object (in employees,,…)
var response = await ElasticSearch.Instance._elasticClient.SearchAsync(q => q
.Index(“demo”)
.Source(so => so
.Includes(f => f
.Field(f => f.CompanyName)
.Field(f => f.Employees.FirstOrDefault().LastName)))
.Suggest(su => su
.Completion(“suggestion”, cs => cs
.Field(f => f.Suggest)
.Prefix(input.CompanyName)
.Fuzzy(f => f
.Fuzziness(Fuzziness.Auto))
.Size(10)
)
)
);
Could you give a bit more context?