Observed since 21 Oct, 11:04 UTC
No superseded metadata states collected. Unchanged observations do not create new revisions.
A batch to analyse AI jobs in Vietnam
This repo contains batch codes to collect data from many top job postings sites in Vietnam such as Indeed VN, VietnamWorks, TopCV, ...
We use Github Actions to collect the data automatically. Please note that, some websites in Vietnam have mechanisms to prevent scrappers like bots, therefore, we must keep retrying every 5 minutes.
Currently, the list of website we are collecting data from is as follows.
| Website | URL | Batch from | Batch cron | Queries |
|---|---|---|---|---|
| TopCV | https://www.topcv.vn | 2023-08-19 | 59 12 * * * or manual | ai engineer, computer vision, machine learning |
| VietnamWorks | https://vietnamworks.com | 2023-08-19 | 59 12 * * * or manual | ai engineer, computer vision, machine learning |
| Indeed Vietnam | https://vn.indeed.com | 2023-08-19 | 59 12 * * * or manual | ai engineer, computer vision, machine learning |
Besides the batch app which is setup in Github Actions to crawl data daily, we provide an online app to test the scenarios of data collected. We use MongoDB to store the data collections. To setup an environment for analysing data, see mongodb environment setup.
To run the online app:
$ python uninstall aijobs_batch
$ python setup.py install
$ aijobs_online --reload --workers 1 --host 0.0.0.0 --port 9000 --log_level info
Download counts and popularity will appear here as the collector receives them.
| Version | Published | Direct dependencies | Open version |
|---|---|---|---|
| 1.0.0a1 | 27 Aug 2023 | 0 |
This describes our collected data; it is not a security assessment.
Shared repository data · GitHub
Observed since 21 Oct, 11:04 UTC
No superseded metadata states collected. Unchanged observations do not create new revisions.
package:026059be8b6820a0866f6edb61dcc35ffe9377424723b033d2f6dcbbf5379066