Hello! My name is Dobromir Nikolov and in this tutorial I’m going to teach you how you can integrate ASP.NET Core together with Elastic and NEST 5.X to build a simple, yet powerful recipes search engine (full code on github). The tutorial is going to be split in 4 parts:
- Installing the NEST package and using it to connect your application to your Elastic instance.
- Implementing functionality for reading and indexing the data into Elastic through NEST.
- Implementing the Search functionality.
- Implementing Autocomplete, MoreLikeThis and GetById functionality.
What you’ll learn
By the end of part 4 you’ll have learned:
- How to enable your .NET application to "talk" to your Elastic instance through NEST’s
ElasticClient
. - How to provide an
ElasticClient
instance throughout your application. - How to use NEST
AttributeMapping
. - How to use the
ElasticClient
object to index data into Elastic. - How to build an Elastic query using C# and
fluent DSL
. - How to make use of the
Completion
andMoreLikeThis
queries.
In this part we’re going to cover the first 2 steps, but first, let’s take a look at…
The application we’re building
The application is simple, but extremely flexible in terms of searching data.
Behind the scenes, the user inputted query is converted to a query_string query that is sent to Elastic. We’re taking advantage of query_string‘s native support for wildcards, boolean operators, phrase matching and boosting.
The user inputted string is proccessed in the following way:
> * Exact phrases to look for are marked with quotes ("an example phrase").
> * Each word/phrase that isn’t marked in any way (example) will be present in the result set’s ingredients.
> * Each word/phrase that is marked with a minus (-example) will not be present in the result set’s ingredients.
> * Each word/phrase can be boosted to become more relevant (score higher) in the search results (example^2).
So, if we want to look for recipes containing garlic, tomatoes, chopped onions and not containing eggs and red-pepper flakes, we’ll have to input the query string:
> garlic^2 tomatoes -egg* "chopped onions" -"red-pepper flakes" // Order doesn’t matter
As you can see, we’ve boosted the garlic. This means that recipes where garlic appears more frequently will score higher.
The asterisk symbol following the egg is called a wildcard query. It means that it’s going to look for words that start with egg and continue with anything else, no matter the length (for example, egg*** is going to match **eggs, eggplant, eggbeater, etc.), very similar to SQL’s like.
You can substitute the asterisk with a "?" sign, which will indicate that Elastic should look for just a single missing letter.
This spits out the result:
Set up Elastic on your local machine
If you haven’t done so already, there is a very nice official guide on setting up Elastic on your local machine. You can check it out here. I advise you to also set up Kibana, since it’s a really useful tool for directly interacting with Elastic. We won’t be relying on Kibana in this tutorial, but I don’t think you can escape it in the long run.
Using NEST to connect to your local cluster
By now, you’ve installed and ran Elastic, but in order to make use of its amazing functionality, we have to be able to communicate with it.
In order to strap up .NET to our Elastic cluster, we’ll be using the NEST client, which is the official .NET client that is supported by Elastic. It’s open-source and you can take a look at the code or contribute on Github. You can learn more about NEST by reading the docs.
Step 1. Create an empty ASP.NET Core project
Just fire up Visual Studio and go to:
> File -> New -> Project -> ASP.NET Core Web Application -> Web API -> No Authentication -> OK
Note that if you are using a version different than 2015, there could be a slight difference.
Step 2. Install NEST
NEST is distributed as a Nuget package, so in order to install it, just open your package.json file and under the "dependencies" section, insert a new line. (in VS 2017, you’ll have to edit the .csproj file)
"NEST": "5.4.0" // Or any 5.x version
Alternatively, you can just run the following command in your package manager console.
>Install-Package NEST
Step 3. Provide an ElasticClient
throughout your application
In order to "talk" to an Elastic instance, NEST uses an object called ElasticClient
.
ElasticClient
is responsible for translating our C# queries to plain JSON requests that are sent to and processed by the Elastic cluster we’re connected to. The client is thread-safe and we’ll have just one instance of it throughout our application.
These are the steps we’re going to take:
- Add our Elastic settings to our appsettings.json file.
- Create an
ElasticConnectionSettings
class, which will serve as a container for the aforementioned settings. - Create an
ElasticClientProvider
class, which will be responsible for instantiating and containing the actualElasticClient
we’ll be using. This is the class we’re going to inject when we need anElasticClient
. - Register the
ElasticClientProvider
class in the DI.
Step 1. Add our Elastic settings to appsettings.json
>Note: You can just go to step 3 and hardcode your cluster’s url and index. I don’t like hardcoding things, that’s why I’m moving my settings to appsettings.json.
Open appsettings.json and insert the following section.
"ElasticConnectionSettings": {
"ClusterUrl": "http://localhost:9200",
"DefaultIndex": "recipes"
}
If you want to connect to another url/use another index, just change the variables.
Step 2. Create the ElasticConnectionSettings
class
In the root of your project, create a folder called Elastic. In this folder we’ll be containing all the Elastic specific objects. Now create a class called ElasticConnectionSettings
with the following implementation:
public class ElasticConnectionSettings
{
public string ClusterUrl { get; set; }
public string DefaultIndex
{
get
{
return this.defaultIndex;
}
set
{
this.defaultIndex = value.ToLower();
}
}
private string defaultIndex;
}
Its properties will be matched against the ones in the appsettings.json file. The DefaultIndex
property has a setter that will always set the lowercase value of the index specified in appsettings.json. This is because indices in Elastic must always be lowercase.
Step 3. Create the ElasticClientProvider
class
In the Elastic folder we created, add a new class called ElasticClientProvider
with the following implementation:
using Microsoft.Extensions.Options;
using Nest;
public class ElasticClientProvider
{
public ElasticClientProvider(IOptions<ElasticConnectionSettings> settings)
{
// Create the connection settings
ConnectionSettings connectionSettings =
// Get the cluster URL from appsettings.json and pass it in
new ConnectionSettings(new System.Uri(settings.Value.ClusterUrl));
// This is going to enable us to see the raw queries sent to elastic when debugging (really useful)
connectionSettings.EnableDebugMode();
if (settings.Value.DefaultIndex != null)
{
// Get the index name from appsettings.json and pass it in
connectionSettings.DefaultIndex(settings.Value.DefaultIndex);
}
// Create the actual client
this.Client = new ElasticClient(connectionSettings);
}
public ElasticClient Client { get; }
}
Now, in order for our provider to work properly, we have to register some services in the Startup.cs file.
Open Startup.cs and in the ConfigureServices
method insert the following lines:
// Get the connection settings from appsettings.json and inject them into ElasticConnectionSettings
services.AddOptions();
services.Configure<ElasticConnectionSettings>(Configuration.GetSection("ElasticConnectionSettings"));
This will tell to our application that we have a section in appsettings.json called ElasticConnectionSettings that we want to be mapped to the ElasticConnectionSettings
class we made earlier.
Step 4. Register the ElasticClientProvider
class for injection
Just below the lines we added in Startup.cs in step 3, add the following:
// Register the client provider as a singleton
services.AddSingleton(typeof(ElasticClientProvider));
This will enable us to inject ElasticClientProvider
wherever we need and get our ElasticClient
from it.
Example usage:
public class SomeClassThatNeedsAClient
{
public SomeClassThatNeedsAClient(ElasticClientProvider provider)
{
_client = provider.Client;
}
private readonly ElasticClient _client;
}
Congratulations!
In just a few simple steps, we managed to create, connect and provide an ElasticClient
throughout our application! In part 2, we’ll be indexing 170000 recipes!
If you have any questions, don’t hesitate to ask in the comments! If you didn’t like something and have suggestions on how to improve, please contact me!
Can you provide link to the next parts
Thank you for this useful blog post .
I Think in the `ElasticClientProvider` class , you should change `settings.Value.Index` to `DefaultIndex` because it does not contain `Index` property .
@uthman Thanks! I’ve changed it.
@abhishek Link to part 2 – https://devadventures.net/2018/04/23/reading-and-indexing-data-in-elasticsearch-using-asp-net-core-and-nest-5/
For the other parts just go to the homepage, you can’t miss them.
I have error like this “Application startup exception: System.NullReferenceException: Object reference not set to an instance of an object.
at XonadonUz.Startup.ConfigureServices(IServiceCollection services) in D:\Xonadon\xonadonuz\Source\XonadonUz\Startup.cs:line 67
— End of stack trace from previous location where exception was thrown —
at Microsoft.AspNetCore.Hosting.ConventionBasedStartup.ConfigureServices(IServiceCollection services)
at Microsoft.AspNetCore.Hosting.Internal.WebHost.EnsureApplicationServices()
at Microsoft.AspNetCore.Hosting.Internal.WebHost.BuildApplication()” in my asp.net core 2.1 web api app,
In Startup.cs:line 67:” services.Configure(Configuration.GetSection(“ElasticConnectionSettings”));”
Please help
Check your `appsettings.json` file. Make sure that you have an “ElasticConnectionSettings” section. See the original file for reference:
https://github.com/dnikolovv/elasticsearch-recipes-nest-angular/blob/master/src/ElasticsearchRecipes/appsettings.json
where are the other parts?
You’ll easily find them by going to the home page – https://devadventures.net/
I’ll be adding links in each article.