Remote Network And System Administration Job In IT And Networking

Elasticsearch performance help - 22K records but fetch time is 6 seconds.

Find more Network And System Administration remote jobs posted recently Worldwide

Problem 1: Performance issue (even though not much data)
We have elasticsearch instance in development server. Facing extremely slow search fetch times and we want to know what we can do to make it fast.
# of records: 22000
Size of index: 980MB (< 1Gb)
Each document has 20-30 fields and 22k docs in total.
.
Use docvalues for all output fields.
Query has size =25,000 takes 6-8 seconds.
Query size=1000, runs faster.
output fields = 6-8 fields are returned (none is nested). All are docvalues.
.
Total query time using curl command: 6-7 seconds to fetch all 22k document with output fields as posted above. We need all 22k documents hence size = 25k
.
Profiled the query time:
Search query time: 2 ms (milli seconds) YES
All time is fetch time.
.
Heapsize = 12gb
Machine has 32gb memory. Enough of free memory left.
swap size = 4gb (none utilized)

.
Problem 2: Few queries on design.
About the recuiter
Member since Mar 14, 2020
Sohal Gupta
from Łódź Voivodeship, Poland

Skills & Expertise Required

Elasticsearch 

Open for hiringApply before - Nov 21, 2024

Work from Anywhere

40 hrs / week

Fixed Type

Remote Job

$76.31

Cost

Offer to work on this project closes in 88 days!
Are you interested in this Opportunity?

Looking for help? Checkout our video tutorial
How to search and apply for jobs

How to apply? Do you have more questions about the Job?
See frequently asked questions

Similar Projects

Add SOPHOS XG firewalls to ELK server

1) Ingest SOPHOS Xg firewall logs into ELK

See https://github.com/enigy/SophosXG-ELK

2) Develop dashboards in ELK that are similar to Sophos Reports

Elasticsearch and Kibana expert

looking for Elasticsearch and Kibana expert to join our team to develop a dashboard, search function

Looking for Hadoop Architect/Expert

I am looking for a Hadoop Sr Architect/Consultant who has some good hands on Spark, Kafka, BI(OpenSource Stack)

Back-end developer

The Cambridge Centre Finance is looking for Backend Developer (2+ years experience) for various tasks including:

- Database design
- Developing tools for sharing research data (API development and documentation)
- Extending Firebase...read more

AWS Serverless Stack Expert Needed

Looking for someone to help with setup/configuration of various serverless components as well as in-depth comparisons/consultations of available options. Good understanding of GCP/AWS serverless stack is required (DynamoDB, Firebase, Appsync, Lambda@...read more