2. The work involved is to use Nifi for log aggregation. Aggregation will involve clubbing logs of a similar type that occur during a defined period (say 1 minute/2 minute etc). This will amount to sending out a single log from Nifi with the count of the number of logs received of that type during the defined period. For example, let us say Nifi receives 10 logs of type User login failure within a period of 1 minute. Then Nifi should send out one log with the count field within the log incremented to 10.
3. Will need to conduct a POC using Nifi consisting of the following:
a. Build in Nifi the log aggregation logic where logs from devices like IPS, Firewalls, etc. are aggregated over a defined criterion
b. Build in capability to change the aggregation criteria externally through a UI - the aggregation criteria will mention the fields to be aggregated and the duration
c. Optimize Nifi to process aggregation of logs at speed of 20000 logs per second
d. Optimize Nifi to run in a cluster to scale up as required to support 20000 logs per second","employmentType":["FULL_TIME","PART_TIME","CONTRACTOR","TEMPORARY","PER_DIEM"],"jobLocationType":"TELECOMMUTE","hiringOrganization":{"@type":"Organization","name":"Toogit","sameAs":"https://www.toogit.com/","logo":"https://www.toogit.com/images/toogit_logo_initial.png"},"directApply":true,"jobImmediateStart":true,"industry":"IT & Software","identifier":{"@type":"PropertyValue","name":"Toogit","value":390397},"skills":["Apache NiFi"],"applicantLocationRequirements":[{"@type":"Country","name":"IN"},{"@type":"Country","name":"Canada"},{"@type":"Country","name":"USA"},{"@type":"Country","name":"Germany"},{"@type":"Country","name":"Pakistan"},{"@type":"Country","name":"Philippines"},{"@type":"Country","name":"Indonesia"},{"@type":"Country","name":"Saudi Arabia"},{"@type":"Country","name":"Nigeria"},{"@type":"Country","name":"Kenya"},{"@type":"Country","name":"Ghana"},{"@type":"Country","name":"China"},{"@type":"Country","name":"Russia"},{"@type":"Country","name":"Bangladesh"}],"validThrough":"2024-12-30T19:09:47+05:30","url":"https://www.toogit.com/freelance-jobs/MzkwMzk3"}
Remote Network And System Administration Job In IT And Networking
Find more Network And System Administration remote jobs posted recently Worldwide
Work from Anywhere
40 hrs / weekHourly Type
Remote Job$19.09
Cost Looking for help? Checkout our video tutorial
How to search and apply for jobs
How to apply? Do you have more questions about the Job?
See frequently asked questions
Must have experience in Analysis, Development, Testing, Implementation, Maintenance and Enhancements on various IT Data Warehousing Projects based on Hadoop Big Data Technologies.
Key Skills Include:
Hadoop Platforms: EMR, Cloudera/Hort...read more
Hello Rockstart Data Engineer,
Thats what we are looking for!
We want to develop highly optimized , scalable and resilient DATA CONNECTORS to fetch operational data on-demand basis. User shall request the data from a web applic...read more
Looking to create a bot to auto place funds or send me a notification alert to place funds when (x) occurs on another website. Similar to those creating bots or trading software to trade stocks and currency following a pattern. Very simple project.