'
-```
+Follow the [instructions](https://github.com/box/box-python-sdk) provided by BoxDev to create a Box application to store input data. To connect to the Box application, modify the Box application settings in `config/config.json` by following these [instructions](https://developer.box.com/guides/authentication/jwt/jwt-setup/#generate-a-keypair-recommended).
-## **Running COCOpen**
+## **Run COCOpen**
-Open the `config/parameters.yaml` file and modify parameters like `dataset_name` (the name of the generated dataset directory), `train_images` (the number of images in the generated training set), and `max_instances` (the maximum number of objects of a particular category per image). Run COCOpen by performing:
+Run COCOpen with
```bash
-# Run COCOpen
-$ ./run.sh
+./run.sh
```
The generated dataset saves to the `datasets` directory under the root directory of this repository. See [example run](https://github.com/RMDLO/COCOpen-OpenCV/blob/main/docs/EXAMPLE_RUN.md) to see a demonstration of generating a simple dataset of ethernet cables and ethernet devices with category, bounding box, and instance segmentation mask annotations in the COCO format.
diff --git a/docs/LEARN_MORE.md b/docs/LEARN_MORE.md
index b9e2b30..cedc013 100644
--- a/docs/LEARN_MORE.md
+++ b/docs/LEARN_MORE.md
@@ -21,12 +21,12 @@ COCOpen performs the following tasks to automatically generate labeled object in
This workflow is shown in the figure below.
- 
+ 
## **Custom Data**
-The UIUCWires dataset stored on Azure is provided to demonstrate COCOpen by default. The dataset includes images of single wire and single devices against a black background. It also includes a folder of background images which are applied to generate background noise in the dataset as a form of data augmentation. Each image type (wire, device, and background) is stored in its own Azure storage container labeled with the object's category. All single-object images were captured with an Intel RealSense d435 camera with 1920x1080 resolution. The table below shows the number of images by object category.
+The UIUCWires dataset stored on Box is provided to demonstrate COCOpen by default. The dataset includes images of single wire and single devices against a black background. It also includes a folder of background images which are applied to generate background noise in the dataset as a form of data augmentation. Each image type (wire, device, and background) is stored in its own folder labeled with the object's category. All single-object images were captured with an Intel RealSense d435 camera with 1920x1080 resolution. The table below shows the number of images by object category.
@@ -38,7 +38,7 @@ The UIUCWires dataset stored on Azure is provided to demonstrate COCOpen by defa
The color contrast between the objects in each image and the background allows for using color thresholding to automatically annotate the original images. A user who wishes to use COCOpen to perform copy-paste augmentation on their own dataset will need to ensure the scene images only contain a single object. The scene for the images fed into COCOpen must also minimize shadowing. In our experience, the scene object can generate a shadow onto itself and the camera apparatus can generate a shadow onto the scene. For best results, we recommend checking lighting conditions before collecting data.
-After users generate their own dataset of single-object RGB images, the images should be uploaded to individual storage containers on Azure. The container should contain only folders of images, and the title of each container of images should correspond to the object category label (all lower-case). See our [Configure Azure Storage Container](https://github.com/RMDLO/COCOpen-OpenCV/blob/main/docs/README_AZURE.md) documentation for more details and to see the directory structure for our container.
+After users generate their own dataset of single-object RGB images, the images should be uploaded to individual folders on Box. The container should contain only folders of images, and the title of each container of images should correspond to the object category label (all lower-case).
## **References**
diff --git a/docs/images/COCOpen.png b/docs/images/COCOpen.png
index 0afb581..d14fce7 100644
Binary files a/docs/images/COCOpen.png and b/docs/images/COCOpen.png differ
diff --git a/src/cocopen.py b/src/cocopen.py
index c35a81b..ffea055 100644
--- a/src/cocopen.py
+++ b/src/cocopen.py
@@ -13,13 +13,13 @@
import numpy as np
import cv2
from pycocotools import mask as pycocomask
-from azure.storage.blob import BlobServiceClient
+from boxsdk import JWTAuth, Client
class COCOpen:
"""
The COCOpen class provides all functions necessary for automatically
- annotating and augmenting a dataset of images stored on Azure.
+ annotating and augmenting a dataset of images stored on Box.
"""
def __init__(
@@ -74,25 +74,26 @@ def __init__(
]
self.category_to_container_client = {}
+ self.category_to_box_folder = {}
self.category_to_train_list = {}
self.category_to_val_list = {}
- def init_azure(
+ def init_box(
self,
) -> None:
"""
- Initializing Azure connection for for accessing blob storage
+ Initialize Box connection for accessing image data
"""
- # Initializing connection with Azure storage account
- conn_str = self.param["directory"]["AZURE_STORAGE_CONNECTION_STRING"]
- blob_service_client = BlobServiceClient.from_connection_string(
- conn_str=conn_str
- )
- # Map category names to blob_service_client of that category
+ # Initialize connection with Box client
+ config_file_path = self.param["directory"]["BOX_CONFIG_FILE_PATH"]
+ self.box_client = Client(JWTAuth.from_settings_file(config_file_path))
+ root_folder = self.box_client.folder("0")
+ # Map category names to box folder object of that category
for category in self.categories:
- self.category_to_container_client[
- category["name"]
- ] = blob_service_client.get_container_client(category["name"])
+ for item in root_folder.get_items():
+ if item.name == category["name"] and item.type == "folder":
+ self.category_to_box_folder[category["name"]] = item
+ break
def make_new_dirs(self) -> None:
"""
@@ -153,38 +154,36 @@ def object_semantics(
def create_image_list(self) -> None:
"""
- Creates foreground image list from images on Azure and
+ Creates foreground image list from images on Box and
splits the list into 'train' and 'val' sets
"""
# Map category names to list of object images of that category
for category in self.categories:
# Creating list of all foreground images of that category
- azure_category_all_list = self.category_to_container_client[
+ box_category_all_list = self.category_to_box_folder[
category["name"]
- ].list_blobs()
+ ].get_items()
# Splitting foreground images into 'train' and 'val' sets
train_list = []
val_list = []
- for blob in azure_category_all_list:
+ for box_image in box_category_all_list:
rand = random.random()
if 0 <= rand < self.param["dataset_params"]["train_split"]:
- train_list.append(blob.name)
+ train_list.append(box_image.id)
else:
- val_list.append(blob.name)
+ val_list.append(box_image.id)
self.category_to_train_list[category["name"]] = train_list
self.category_to_val_list[category["name"]] = val_list
- def download_image_from_azure(self, img, category):
+ def download_image_from_box(self, img_id):
"""
- Downloads image from Azure blob storage
+ Downloads image from Box storage
"""
- # Get blob and read image data into memory cache
- blob = self.category_to_container_client[category].get_blob_client(img)
- img_data = blob.download_blob().readall()
- # Convert image data to numpy array
- img_array = np.asarray(bytearray(img_data), dtype=np.uint8)
+ # Get file and read image data into memory cache
+ file = self.box_client.file(img_id)
+ file_data = file.content()
# Decode image
- src = cv2.imdecode(img_array, cv2.IMREAD_COLOR)
+ src = cv2.imdecode(np.frombuffer(file_data, np.uint8), cv2.IMREAD_COLOR)
return src
@@ -212,7 +211,7 @@ def contour_filter(self, frame, contour_max_area=2075000):
def generate_supercategories(self):
"""
Generate dictionary for super categories based on param .yaml file.
- Super categories are used when an object category, like "wire", may
+ Super categories are used when an object category, like "single-wire", may
contain subcategories, like "ethernet" or "power."
"""
for key in self.param["categories"]:
@@ -223,12 +222,12 @@ def generate_supercategories(self):
}
self.categories.append(supercategory_dict)
- def get_object_info(self, img, category):
+ def get_object_info(self, img_id, category):
"""
This function returns the image array and the mask array
"""
# Get mask
- src = self.download_image_from_azure(img=img, category=category)
+ src = self.download_image_from_box(img_id=img_id)
median = cv2.medianBlur(src, 9) # Blur image
gray = cv2.cvtColor(median, cv2.COLOR_BGR2GRAY) # Convert to grayscale
_, mask = cv2.threshold(
@@ -483,7 +482,6 @@ def random_flip(self, total_num_instances, all_img_arr, binary_mask_arr):
Random flips
"""
for m in range(0, total_num_instances):
-
horizontal = int(2 * random.random())
vertical = int(2 * random.random())
@@ -502,9 +500,9 @@ def add_random_background(self, image_list, mask_array_1):
Add random background
"""
randbg = int(random.random() * len(image_list["background"]))
- bg_img = image_list["background"][randbg]
- # Download background image from azure
- src = self.download_image_from_azure(img=bg_img, category="background")
+ bg_img_id = image_list["background"][randbg]
+ # Download background image from box
+ src = self.download_image_from_box(img_id=bg_img_id)
bg_arr = src.astype("uint8")
# Exception: background image dimensions!=image shape parameter
@@ -547,7 +545,6 @@ def combine_operations(
Perform combine operations
"""
for i in tqdm(range(num_images)):
-
# scale factor for this image
scale = scale_range[0] + random.random() * (scale_range[1] - scale_range[0])
@@ -612,7 +609,6 @@ def combine_operations(
# update coco info
for mask1 in binary_mask_arr[randint]:
-
mask_array_1.append(mask1)
msk1 = np.zeros((self.height, self.width, 3)).astype("uint8")
diff --git a/src/run.py b/src/run.py
index a438af7..a51df7d 100644
--- a/src/run.py
+++ b/src/run.py
@@ -26,7 +26,6 @@ def main(params):
# Generate a new COCO-formatted dataset
if parameters["generate_dataset"]:
-
# initialize cocopen object
cocopen = COCOpen(
param=params,
@@ -35,8 +34,8 @@ def main(params):
# Create categories dictionary from parameters
cocopen.generate_supercategories()
- # Initializing Azure connection
- cocopen.init_azure()
+ # Initializing Box connection
+ cocopen.init_box()
# Make new directories
cocopen.make_new_dirs()
diff --git a/src/train.py b/src/train.py
index be57c34..cc1bbeb 100644
--- a/src/train.py
+++ b/src/train.py
@@ -225,7 +225,6 @@ def __init__(
self,
parameters: dict,
) -> None:
-
self.parameters = parameters
# COCO Dataset Loading
@@ -284,4 +283,4 @@ def predict(self):
f"./train/opencv/{self.name}_{self.model}",
os.path.basename(ann["file_name"]),
)
- cv2.imwrite(save_directory, out.get_image())
\ No newline at end of file
+ cv2.imwrite(save_directory, out.get_image())
+ 
