i-love-data

General approach on collecting custom image datasets for classification purpouses

1. Search for prepared datasets:

• Kaggle dataset search
• CoCo Dataset
• ImageNet dataset
• Google dataset search
• Google Images Download Python Script

2. Search for prepared datasets:

• Get youtube video and cut it on frames
• Use image search API
• Azure cognitive services (Bing Web Search API)
• DuckDuckGo (DuckDuckGo Search API)

3. Awesome links:

• Pyimagesearch - blog about image processing and machine learning
• How to collect your deep learning dataset
• How to create dataset using API
• Remove Image Background Service
• Create Anime Characters with AI
• Increase Image Resolution Service (via Convolutional neural network)
• Fake faces images generator (thispersondoesnotexist)

ML links:

• XGBoost - Gradient Boosting Library
• SarcasmDetection using Neural Network
• Style Transfer Network (NVIDIA Corporation)

• DeepPavlov framework (Text QA, ODQA, Ranking, Entity recognition, Intent classification, Insult detection)

• YOLO (Real-Time Object Detection)

4. Tool for object detections:

• Classical problem is when you can google for some pictures of your interest (like "can of cola") and the picture you find contains several objects (different cans). So you need to somehow preprocess source image and extract regions of your interest. This is where pretrained YOLO or Retina may be useful (or your custom last trained dense layer using transfer learning). Implementation can be found here

• Instead of using the raw library we had written a simple wrapper that accepts directories with pictures and target classes you want to extract. Assuming you want to extract cans from some folder with pictures:

After we apply wrapper to the source image we can get seaparate cans and after it feed it directly to the network.

5. Annotation:

• labelme is a super simple python module with gui for any annotation task: classification, segmentation, bb-boxes and more.