1) store data scraped into either dynamodb
2) listen to notifications to fire news scrape
3) being able to run cronjobs
4)being able to unify all time to one timezone
5) being able to use a proxy to prevent ip blocking ( will purchase and provide)
6) 3 lambdas: *one for news,
*one for stats,table,fixtures,googlesheet
*another for team fixtures
*The required are 8 apis end points,
*6 simple html page scripts from scratch and one to be duplicated with different url
*one rss feed listener, and one to be duplicated with different url
*two apis response scraping scripts
the project is divided into these steps:
* news:
1)scripts: there are 5 rss feeds created, there is one should be created, and another one duplicated with different url. All these rss feeds should run in one lambda every 5 minutes and if there are news urls open the html content and scrape it.
2) 4 simple html pages scraping to run if there were new news posted in the rss feeds
3) db table is already done only missing isLocal coloumn
4) 2 apis, one with paging
*Standing:script,db is created ( should run every 10 minutes from 1 pm to 12 am)
1)1 api to create
2) add a column to detect if the team position increased(1) or decreased(-1) ,or remained the same(0) from te previous scrape insert
*Fixtures: scraping api is done, db is done
need to create one api end points
Stats html page, googlesheet scrape and team fixtures will be done from scratch
check pdf file for more details
start job offer with RD07
preferably to use python for scraping as most scripts are done with python","employmentType":["FULL_TIME","PART_TIME","CONTRACTOR","TEMPORARY","PER_DIEM"],"jobLocationType":"TELECOMMUTE","hiringOrganization":{"@type":"Organization","name":"Toogit","sameAs":"https://www.toogit.com/","logo":"https://www.toogit.com/images/toogit_logo_initial.png"},"identifier":{"@type":"PropertyValue","name":"Toogit","value":353514},"skills":["API Development","AWS Lambda","Python","Web Scraping"],"applicantLocationRequirements":[{"@type":"Country","name":"IN"},{"@type":"Country","name":"Canada"},{"@type":"Country","name":"USA"},{"@type":"Country","name":"Germany"},{"@type":"Country","name":"Pakistan"},{"@type":"Country","name":"Philippines"},{"@type":"Country","name":"Indonesia"},{"@type":"Country","name":"Sri Lanka"},{"@type":"Country","name":"Nigeria"},{"@type":"Country","name":"China"},{"@type":"Country","name":"Russia"},{"@type":"Country","name":"Bangladesh"}],"validThrough":"2024-09-19T09:58:16+05:30","url":"https://www.toogit.com/freelance-jobs/MzUzNTE0"}
Remote Web Development Job In IT And Programming
Find more Web Development remote jobs posted recently Worldwide
Work from Anywhere
40 hrs / weekFixed Type
Remote Job$623.35
Cost Looking for help? Checkout our video tutorial
How to search and apply for jobs
How to apply? Do you have more questions about the Job?
See frequently asked questions
We are looking for subject matter experts to review our Big Data Engineer interview screen tests: Spark-Python and related topics. These questions will be used as interview questions in a hiring process and we need to be sure that the problem stateme...read more
Looking for an experienced Python developer with functional knowledge in stock trading. He/she should be able to
- Understand the basics of stock trading / Equities
- Data mining, Data science experience.
Well be using Quantopian to...read more
Were looking for an expert Computer vision developer for long term who is comfortable on working on both Server (testing models), and mobile (implementing via TFlite and CoreML).
** Only apply if youre experienced in TensorFlow Lite
**...read more
I know very little about programming and want to learn programming enough that I understand and can operate on my own.
This will be an ongoing project to learn Programming:
I would like a plan created with assignments.
One on on...read more
Parse data from 6 PDFs and 3 Excel sheets into a specific format in Excel.
I will share all data files with you, once you have shortlisted for this task.
Those file shows what the final columns should be, examples of entries and data that ne...read more