Remote Network And System Administration Job In IT And Networking

Elasticsearch performance help - 22K records but fetch time is 6 seconds.

Find more Network And System Administration remote jobs posted recently Worldwide

Problem 1: Performance issue (even though not much data)
We have elasticsearch instance in development server. Facing extremely slow search fetch times and we want to know what we can do to make it fast.
# of records: 22000
Size of index: 980MB (< 1Gb)
Each document has 20-30 fields and 22k docs in total.
.
Use docvalues for all output fields.
Query has size =25,000 takes 6-8 seconds.
Query size=1000, runs faster.
output fields = 6-8 fields are returned (none is nested). All are docvalues.
.
Total query time using curl command: 6-7 seconds to fetch all 22k document with output fields as posted above. We need all 22k documents hence size = 25k
.
Profiled the query time:
Search query time: 2 ms (milli seconds) YES
All time is fetch time.
.
Heapsize = 12gb
Machine has 32gb memory. Enough of free memory left.
swap size = 4gb (none utilized)

.
Problem 2: Few queries on design.
About the recuiter
Member since May 20, 2018
Sridhar
from Tartu, Estonia

Skills & Expertise Required

Elasticsearch 

Open for hiringApply before - Nov 1, 2024

Work from Anywhere

40 hrs / week

Fixed Type

Remote Job

$76.42

Cost

Offer to work on this project closes in 101 days!
Are you interested in this Opportunity?

Looking for help? Checkout our video tutorial
How to search and apply for jobs

How to apply? Do you have more questions about the Job?
See frequently asked questions

Similar Projects

Data Engineer is needed for setting up our Data warehousing in AWS

We are looking for a consultant/freelancer that could help us with the following:

- Consulting on how to structure our data analytics database so that is scalable and performant.
- Consulting on the right pipeline of the data given our...read more

Solution architect for web advertising platform

We are building a Consumer, Services Provider marketplace for web/mobile. Tech stack currently includes WordPress, Angular, MongoDB, MySQL, etc. Looking for an experienced solution architect to partner with business and dev teams to help with:
read more

MediaWiki search optimisation, elasticsearch

We have a MediaWiki installation on a Ubuntu linux box on AWS, with a MySQL database on AWS. Weve uploaded a lot of articles to the database and installed the plugins and extensions to the MediaWiki code.

The problem now is the search feat...read more

Looking for PHP architect having strong eCommerce domain experience

e-Commerce SME Solution Architect

We are looking for an innovative Solution Architect to develop and implement systems architecture that will meet business needs. You should have in-depth understanding of any suitable PHP framework that can...read more

Multichannel (EMAIL + SMS) Platform requires Expert Consultant for tech stack definition

Hello,

We need help from an expert (with background in MarTech - Marketing Technology, Big Data , ML ) for defining the tech stack and platform architecture that will allow to:

1. Import and manage millions of contacts at once, with...read more