How To Use Model Assisted Labeling To Boost Annotation Performance With Supervisely

Smart Tool: What is it?

Smart Tool is a high-powered interactive segmentation tool. Its functionality is based on the creation of a mask predicted by artificial intelligence (AI). This unique tool occupies a special place in the Supervisely arsenal, significantly accelerating the annotation process.

What makes the Smart Tool special? The Smart Tool provides users with access to a diverse range of neural network algorithms for object segmentation task integrated into Supervisely. These include robust models such as RITM, Segment Anything and others, and we are constantly expanding our Ecosystem by integrating new models. It's crucial to note that the quality, accuracy, and speed of segmentation heavily rely on the chosen model.

Smart Tool is particularly useful for tasks that involve selecting areas with complex contours or a large number or variety of objects. Its ability to perform such tasks in a matter of seconds makes it highly effective. Where the model predictions shall be amended, a combination with such complementary tools as Brush and Pen is beneficial. These tools offer the ability to correct imperfections in the masks created by the neural network. The synergy between the Smart Tool and other Annotation instruments enhances the overall versatility and robustness of the annotation process.

This makes Smart Tool one of the most popular annotation tools in Supervisely. Smart Tool may be utilized in almost any domain, including:

• Urban Planning and Infrastructure: the Smart Tool is great for marking up urban scenes and infrastructure designs. Whether you're involved in city planning or site development, it is excellent at accurately annotating structures, roads, and utilities.
• Medical: the Smart Tool facilitates the precise delineation of organs, tissues, and anomalies, thereby contributing to accurate diagnostics and comprehensive treatment planning and also helps to accurately segment medical instruments, which contributes to the development of robotic surgery.
• Agriculture: farmers may rely on AI tools to assess crop health, plan harvests efficiently, and minimize waste through targeted interventions, enhancing overall agricultural productivity. The Smart Tool enables precise annotation of fruits, vegetables, weeds, crops, pests and much more, significantly improving the quality assessment of agricultural produce.
• Automotive: The Smart Tool is really helpful for showing and dividing car damage, like dents and chips. It's also good at finding spots where the tires are worn, giving a better idea of how the vehicle equipment is overall.
• Infrastructure Inspection: In infrastructure inspection, the Smart Tool can be employed to detect damage and hazards on roads, such as potholes or obstructions. This is particularly critical for the safety of autonomous vehicles, which leverage this information to navigate safely and avoid potentially dangerous areas.

Video Tutorial for Smart Tool in the New Labeling Toolbox

Master the efficient utilization of our Smart Tool through a concise 5-minute video tutorial, featuring explicit guidance from our Supervisely expert. Familiarize yourself with the updated interface and explore the functionalities within our annotation toolbox with practical, hands-on experience.

Smart Tool: key features

You can try out our legendary Smart Tool for free directly on the Supervisely Platform!

1. Interact with the Model for Precise Predictions

Creating a mask with the Smart Tool is simple:

1. Click anywhere in the scene to make the top point of the rectangle.

2. Then click again for the bottom point in the opposite direction.

3. The neural network will attempt to identify the main object within that rectangle.

4. To adjust the neural network's predictions, place and drag positive 🟢 and negative 🔴 points around the entire area inside the bounding boxes.

   Click and put a positive point on the part of the object you want IN the final mask. For parts of the background you want to leave OUT of the final mask, put a negative point by clicking.

5. Finalize the process by pressing the SPACE hotkey.

Pro tip: The bounding box around an object doesn’t need to be precisely close to the object’s boundaries. We recommend placing the initial bounding box with about a 10% padding from the object boundaries. This way, the model will have more context, and segmentation will be more accurate near the object boundaries.

2. Segmentation of Complex Objects Made Easy

Regardless of the number or complexity of objects in the image, Smart Tool can manage them with its neural network algorithms. The accuracy and satisfactory performance of the algorithm are significantly influenced by your involvement in editing the mask.

3. One-click Segmentation

Thanks to the accuracy provided by the Smart Tool, objects with simple shapes often require no further adjustments. Simply create bitmap masks and enjoy!

4. Smooth Manual Editing

Neural networks can sometimes struggle with objects they were not trained on or due to other factors. If you need to manually edit the Smart Tool predictions, you can use the Brush, Eraser and Pen tools to remove unnecessary parts or add new ones.

To add a part of the object, select the shape and draw the missing part. To delete a part of the object, press and hold the SHIFT key while repeating the previous steps. The algorithm is the same for both tools.

5. Fast Right-click Object Class Changing

For a seamless transition between object classes, initiate a right-click on the selected item and choose the desired option from the pop-up list. Beyond class changes, this menu provides convenient options for deleting or hiding an object, further streamlining your annotation workflow.

6. Switching Between Models

Neural Networks have various designs and were trained on diverse data, leading to different performances with different types of data. In some cases, it might be helpful to use different neural network models for the Smart Tool within the same dataset.

Switching between interactive segmentation models for the Smart Tool is easy. You can use the ID button in the left toolbar.

Switch between interactive object segmentation models

On our Community instance at Supervisely, we prefer to integrate ClickSEG by default as we see it as the best solution for many different tasks. At the same time, our users can run any pre-trained model from an interactive segmentation category (like Segment Anything, RITM, etc.) on their computers with GPU or train and deploy interactive segmentation model on custom training dataset, creating unique customization opportunities. The whole process is done in a few simple steps, making it as convenient and accessible as possible.

Our enterprise customers have the privilege of selecting their interactive segmentation models through a convenient administration panel and sharing it with all users on their private instance. To get the best experience of the Supervisely platform check out our blog post for more details about how to train a custom interactive segmentation model (plants segmentation use case in agriculture) and deploy it on your server.

How to run any serving App

Train models the way you need

Popular neural network models, such as Segment Anything, were trained on a large portion of data; however, these models are general-purpose and may face difficulties with specific tasks, such as segmenting agricultural images in low quality or detecting road cracks.

In Supervisely, you have the capability to train the Smart Tool specifically for your data directly within the platform, and the best part is, that no coding is required! We're actively experimenting with this feature and are impressed by the results. Feel free to explore other blog posts dedicated to this topic.

• How to Train Smart Tool for Precise Cracks Segmentation in Industrial Inspection

• Automate manual labeling with custom interactive segmentation model for agricultural images

• Unleash The Power of Domain Adaptation - How to Train Perfect Segmentation Model on Synthetic Data with HRDA

• Lessons Learned From Training a Segmentation Model On Synthetic Data

Visualization Settings

Tailor your settings to suit your needs by adjusting opacity levels, selecting random colors for objects while annotating and many other useful features.

Hotkeys

Control the Smart tool 0 more efficiently with HOTKEYS.

Detect object inside BBox - CLICK LBM + CLICK LBM
Add positive point - CLICK LBM
Add negative point - SHIFT + CLICK LBM
Remove feedback point - ALT + CLICK LBM
Drag rectangle - ALT + HOLD LBM
Drag rectangle - ALT + ARROW KEYS
Finish & start new - SPACE

LBM stands for Left Mouse Button

Use ID button to change model. You can deploy more interactive segmentation models in Ecosystem.
Deployed model: Example - Serve ClickSEG#47962.

Scene Navigation
Zoom with MOUSE WHEEL. Hold RIGHT MOUSE BUTTON to move scene.

Conclusion

In summary, the Smart Tool is a fantastic AI tool for precise segmentation. It's versatile and easy to use, making it popular in different fields. Try it for free on the Supervisely Platform and see how it can upgrade your object segmentation tasks.