INTL
Freelancer
전문가
외주
원격 가능
AWS Deployment & Application Optimization Project
예산
$15~$25 CAD
예상 기간
3~6개월
난이도
전문가
기술 스택
Java
Linux
Cloud Computing
Software Architecture
Amazon Web Services
Application Performance Monitoring
Automation
API Development
AWS EC2
Distributed Systems
Concurrency Control
API Error Handling
Performance Tuning
Monitoring & Logging
Telegram Integration
Proxy Management
WebSocket
AI 분석 요약
이 프로젝트는 AWS에 배포된 애플리케이션의 성능을 최적화하고, API 호출 지연을 제거하며, 200ms 미만의 낮은 지연 시간을 달성하는 것을 목표로 합니다. 429, 403, 504와 같은 API 오류를 해결하고, 다중 계정 생성 및 동적 후보 전환 로직을 포함한 복잡한 자동화 파이프라인을 구축하여 안정적인 작업 처리 및 실시간 슬롯 예약을 구현해야 합니다. AWS 성능 튜닝, 분산 시스템 설계 및 API 오류 처리 역량이 필수적입니다.
프로젝트 원문 설명
1- Update application calls should have zero delay so that calls go at full speed.
2- Latency is very important — locally it's 250-300ms, so after deploying on AWS it should be minimum 200ms. So delay = 0 and latency should be low so calls hit fast.
3- We need to pick jobs, and for that we need to eliminate 429, 403, and 504 errors. Why does 429 come? Because we reuse the same token too fast. 403 comes when multiple calls hit from the same IP or combination calls hit together. 504 doesn't come that often, so for that we can keep a retry mechanism.
4- To eliminate 429, we need to use multiple tokens, because a job will only be picked when calls go without even a single 429, with latency 200-250ms and delay 0.
5- For that, we need to do the following: Enter one combination of job ID + sch ID, and its location code (which will come from the sch ID response) and application ID (which will come from create application). We'll add those two manually. Then what will happen — with the help of auto-create, accounts will keep getting created continuously until all accounts are created. Like if I enter 200, it will create 200 accounts.
6- What will the pipeline do? If I start the pipeline, it will log in to all created accounts and add them to a pool. Like if I enter 300, it will log in to 300 accounts. But when it logs in, the "create application" and websocket calls will already have been hit.
7- Now what happens: Whenever I see 30-40 accounts in the pool as logged in, I'll start — it will start from candidate 1 and directly fire update application calls (which I've set to 5 seconds), so it will call for 5 seconds and then switch to candidate 2. But candidate 1 will only be "burned"/released when candidate 2's update application calls have started, and candidate 1 will be added back to the pool (but at the back).
8- So candidates always switch like 2→3→4→5→6... if it reaches 200, it starts again from 1. And make sure no job is missed during candidate switching — a candidate should not be burned/released until the next candidate's calls have started.
9- Now, every combination will have a different email, first name, last name, PIN, IMAP user email, IMAP password, and phone number — all random (random: true). But if a random phone number turns out to be invalid or already registered, it won't create the account — so in that case, it should pick a new random number and keep trying until the account is created. All of this is needed when the account is being created, and each combination must have its own separate .config.json file.
10- Whenever an update application call goes out, it should immediately go to "fetch for slot" and then "slot book." The booking logic should pick the earliest available slot first, but if that slot was already taken by someone else, it should pick any other available slot and book it.
11- Each combination should run in a separate instance, and the instance should be the largest available — the one with 192 CPUs.
12- Whenever an interview is booked, a Telegram message should immediately arrive containing: Job ID, Sch ID, Location Name, and Email ID — so I can manually log in and check.
13- The same process should work for every combination — the same cycle.
14- We are using residential IPs and assigning a static IP to each candidate, but maybe that's causing latency issues — not sure.
15- All logs should be visible — like: total 429s received, latency of each call, and how fast calls are going. This should run continuously and never close until I manually stop it.
2- Latency is very important — locally it's 250-300ms, so after deploying on AWS it should be minimum 200ms. So delay = 0 and latency should be low so calls hit fast.
3- We need to pick jobs, and for that we need to eliminate 429, 403, and 504 errors. Why does 429 come? Because we reuse the same token too fast. 403 comes when multiple calls hit from the same IP or combination calls hit together. 504 doesn't come that often, so for that we can keep a retry mechanism.
4- To eliminate 429, we need to use multiple tokens, because a job will only be picked when calls go without even a single 429, with latency 200-250ms and delay 0.
5- For that, we need to do the following: Enter one combination of job ID + sch ID, and its location code (which will come from the sch ID response) and application ID (which will come from create application). We'll add those two manually. Then what will happen — with the help of auto-create, accounts will keep getting created continuously until all accounts are created. Like if I enter 200, it will create 200 accounts.
6- What will the pipeline do? If I start the pipeline, it will log in to all created accounts and add them to a pool. Like if I enter 300, it will log in to 300 accounts. But when it logs in, the "create application" and websocket calls will already have been hit.
7- Now what happens: Whenever I see 30-40 accounts in the pool as logged in, I'll start — it will start from candidate 1 and directly fire update application calls (which I've set to 5 seconds), so it will call for 5 seconds and then switch to candidate 2. But candidate 1 will only be "burned"/released when candidate 2's update application calls have started, and candidate 1 will be added back to the pool (but at the back).
8- So candidates always switch like 2→3→4→5→6... if it reaches 200, it starts again from 1. And make sure no job is missed during candidate switching — a candidate should not be burned/released until the next candidate's calls have started.
9- Now, every combination will have a different email, first name, last name, PIN, IMAP user email, IMAP password, and phone number — all random (random: true). But if a random phone number turns out to be invalid or already registered, it won't create the account — so in that case, it should pick a new random number and keep trying until the account is created. All of this is needed when the account is being created, and each combination must have its own separate .config.json file.
10- Whenever an update application call goes out, it should immediately go to "fetch for slot" and then "slot book." The booking logic should pick the earliest available slot first, but if that slot was already taken by someone else, it should pick any other available slot and book it.
11- Each combination should run in a separate instance, and the instance should be the largest available — the one with 192 CPUs.
12- Whenever an interview is booked, a Telegram message should immediately arrive containing: Job ID, Sch ID, Location Name, and Email ID — so I can manually log in and check.
13- The same process should work for every combination — the same cycle.
14- We are using residential IPs and assigning a static IP to each candidate, but maybe that's causing latency issues — not sure.
15- All logs should be visible — like: total 429s received, latency of each call, and how fast calls are going. This should run continuously and never close until I manually stop it.
Freelancer에서 원본 확인
원본 보기