Clash Royal Object Detection

Contents

from pykubegrader.tokens.validate_token import validate_token
validate_token('type the key provided by your instructor here', assignment = 'week9-lecture')

validate_token(assignment = 'week9-lecture')

# You must make sure to run all cells in sequence using shift + enter or you might encounter errors
from pykubegrader.initialize import initialize_assignment

responses = initialize_assignment("4_clashroyal", "week_9", "lecture", assignment_points = 73.0, assignment_tag = 'week9-lecture')

# Initialize Otter
import otter
grader = otter.Notebook("4_clashroyal.ipynb")

Clash Royal Object Detection#

In this guideded tutorial we will walk through the process of detecting objects in an image using the scikit-image library.

We will start by loading the image and converting it to grayscale.

Visualizing the images#

We will start by visualizing the two images that we will be using for this tutorial. The first image is a background image and the second image a battle scene between Professor Agar and a random Foe.

Having the background image allows us to detect objects that are not part of the background.

# Import necessary libraries
# Scikit-image is a Python library for image processing. We will use the io, color, filters, feature, measure, and morphology modules.
# from scipy import ndimage as ndi
# matplotlib and numpy using common aliases plt and np
...

# Load the Clash Royale image
# use the io module to load the image, from the assets/figures directory
# the background image is clash1.jpeg and the image to detect is clash2.jpeg
...


# If the image has 4 channels (RGBA), convert it to RGB by dropping the alpha channel
# hint, you can use the [..., :3] syntax to drop the alpha channel
...

# Visualize the images
# build a figure with two subplots, one for the background and one for the image
# use the figsize parameter to set the size of the figure to 10x5
# use the axis parameter to turn off the axis labels
# use the imshow function to display the images
# use the title function to set the title of the subplots, the title of the first image is "Background" and the title of the second image is "Image"
# use the show function to display the figure
...

plt.show()

grader.check("Clash-Royal-Object-Detection-loading-images")

# Convert the images to grayscale
# use the color module method rgb2gray to convert the images to grayscale, assign the results to image_gray and background_gray
...

# Apply edge detection filters
# use the filters module method sobel to apply the sobel edge detection filter to the background_gray and image_gray images, assign the results to edges_sobel_background and edges_sobel_image
# use the filters module method scharr to apply the scharr edge detection filter to the background_gray and image_gray images, assign the results to edges_scharr_background and edges_scharr_image
# use the feature module method canny to apply the canny edge detection filter to the background_gray and image_gray images, assign the results to edges_canny_background and edges_canny_image
...

# Combine edge filters for improved segmentation
# use the numpy module method add to combine the edges_sobel_background, edges_scharr_background, and edges_canny_background images, assign the result to combined_edges_background
# use the numpy module method add to combine the edges_sobel_image, edges_scharr_image, and edges_canny_image images, assign the result to combined_edges_image
...

# Visualize the combined edge filters
# use the matplotlib module method figure to create a figure with two subplots, one for the background and one for the image, make the figure size 10x5
# use the matplotlib module method subplot to create the subplots, the first subplot is for the background and the second is for the image
# use the matplotlib module method imshow to display the combined_edges_background and combined_edges_image images
# use the matplotlib module method title to set the title of the subplots, the title of the first subplot is "Background" and the title of the second subplot is "Image"
# use the matplotlib module method show to display the figure
fig, ax = plt.subplots(1, 2, figsize=(10, 5))

ax[0].imshow(combined_edges_background, cmap="gray")
ax[0].set_title("Background")
ax[0].axis("off")

ax[1].imshow(combined_edges_image, cmap="gray")
ax[1].set_title("Image")
ax[1].axis("off")

plt.show()

grader.check("Edge-Detection-and-Thresholding")

# Compute the difference between the combined edge detection for the background and image
# use the numpy module method abs to compute the absolute difference between the combined_edges_background and combined_edges_image arrays, assign the result to edge_diff
...

fig, ax = plt.subplots(figsize=(10, 5))
ax.imshow(edge_diff, cmap="gray")
ax.set_title("Edge Difference")
ax.axis("off")
plt.show()

grader.check("edge-difference")

# Thresholding to create a binary mask, where the threshold is 0.8
...

fig, ax = plt.subplots(1, 1, figsize=(10, 5))
ax.imshow(binary_mask, cmap="gray")
ax.set_title("Binary Mask")
ax.axis("off")
plt.show()

grader.check("binary-mask")

# Perform morphological closing to close small gaps in the edges
# use the morphology module method closing to perform morphological closing on the binary_mask image, assign the result to closed_mask
# use the morphology module method square to create a square structuring element with a radius of 5
...

fig, ax = plt.subplots(1, 1, figsize=(10, 5))
ax.imshow(closed_mask, cmap="gray")
ax.set_title("Closed Mask")
ax.axis("off")
plt.show()

plt.show()

grader.check("closing-mask")

# Merge connected shapes using morphological dilation
# use the morphology module method dilation to perform morphological dilation on the closed_mask image, assign the result to merged_mask
# use the morphology module method square to create a square structuring element with a radius of 10
...

# Visualize the merged mask
fig, ax = plt.subplots(1, 1, figsize=(10, 5))
ax.imshow(merged_mask, cmap="gray")
ax.set_title("Merged Mask")
ax.axis("off")
plt.show()

grader.check("meger-mask")

# Remove small objects (noise) from the binary mask
# use the morphology module method remove_small_objects to remove small objects from the merged_mask image, assign the result to cleaned_mask
# use the min_size parameter to set the minimum size to 5000, this is in pixels.
...

fig, ax = plt.subplots(1, 1, figsize=(10, 5))
ax.imshow(cleaned_mask, cmap="gray")
ax.set_title("Cleaned Mask")
ax.axis("off")
plt.show()

plt.show()

grader.check("clean-mask")

# from skimage import io, measure, color
# from skimage.measure import regionprops
...

# Threshold the image to create a binary mask
# create a binary mask from the cleaned_mask image, where the threshold is 0.8, assign the result to binary_mask
...

# Label connected regions in the binary image
# use the measure module method label to label the connected regions in the binary_mask image, assign the result to labels
# use the connectivity parameter to set the connectivity to 2
...

# Get properties of labeled regions
# use the regionprops function to get the properties of the labeled regions in the labels image, assign the result to regions
...

# Plot the segmented image
# use the matplotlib module method figure to create a figure with a single subplot, make the figure size 10x10
# use the matplotlib module method subplot to create the subplot
# use the matplotlib module method imshow to display the color.label2rgb(labels, bg_label=0) image
# use the matplotlib module method title to set the title of the subplot, the title is "Segmented Image with Bounding Boxes"
# use the matplotlib module method axis to turn off the axis labels
# use the matplotlib module method show to display the figure
...

# Draw bounding boxes around each labeled region
# use a for loop to iterate over the regions
# use the region.bbox attribute to get the coordinates of the bounding box
# use the matplotlib module method Rectangle to draw the bounding box on the image
# use the matplotlib module method add_patch to add the bounding box to the image
# use the matplotlib module method show to display the figure
...
    # Get the coordinates of the bounding box
    # use the region.bbox attribute to get the coordinates of the bounding box, assign the result to bbox
    # use the region.bbox attribute to get the coordinates of the bounding box, assign the result to bbox assign the results to min_row, min_col, max_row, max_col
    ...

    # Draw the rectangle on the image
    # use the matplotlib module method Rectangle to draw the bounding box on the image, assign the result to rect
    # use the edgecolor parameter to set the edge color to red
    # use the facecolor parameter to set the face color to none
    # use the linewidth parameter to set the line width to 2
    # the rectangle takes the coordinates (min_col, min_row) and the width and height of the rectangle (max_col - min_col, max_row - min_row)
    ...
    
    # use the matplotlib module method add_patch to add the bounding box to the image
    # use the ax.add_patch method to add the bounding box to the image
    ...

plt.show()

grader.check("segment-image-regions")

Submitting Assignment#

Please run the following block of code using shift + enter to submit your assignment, you should see your score.

from pykubegrader.tokens.validate_token import validate_token
validate_token(assignment = 'week9-lecture')

from pykubegrader.submit.submit_assignment import submit_assignment

submit_assignment("week9-lecture", "4_clashroyal")