How to use the ControlNet pre-processor nodes with sample images to extract image data. Includes sample worfklow ready to download and use.

Requirements

• ComfyUI Manager: Recommended to manage plugins.
• ControlNet Auxiliary Preprocessors: Provides nodes for ControlNet pre-processing.
• Sample image to extract data with ControlNet.

Workflow explained

Whenever this workflow is run, the sample image will be enhanced and processed to extract the corresponding data using these nodes:

1. Canny Edge
2. HED soft-edge Lines
3. Depth Anything
4. Scribble Lines
5. Anime Lineart
6. Realistic Lineart
7. DWPose Estimator

The generated images will be saved in a folder with the followig structure: output/YYYY-MM-DD/controlnet/*.png

ControlNet Preprocessors workflow explained

1. Use the Load Image node to open the sample image that you want to process.
2. Set the output image size as follows:
   • The Output Width should be 512 or 768 for SD1.5, 1024 or more for SDXL.
   • The Output Height should be 512 or 768 for SD1.5, 1024 or more for SDXL.
3. Configure the Enhanced and Resize Hint Images resize_mode to Crop and Resize (recommended) or Just Resize. This method could stretch the image, overriding its original aspect ratio, depending on your chosen width and height and the source image’s size.
4. Use the AIO Aux Preprocessor node to optionally apply any other pre-processing method you want to the source image.
5. In the Canny Edge, HED Soft-Edge Lines, Depth Anything, Scribble Lines, Anime Lineart, Realistic Lineart and DWPose Estimator nodes, the output width will be set by the value set from your Output Width primitive node.
6. Configure the DWPose Estimator node as follows:
   1. detect_hand: enable or disable. The sample workflow is set to disable.
   2. detecy_body: enable or disable.
   3. detect_face: enable or disable. The sample workflow is set to disable.
7. The Save Image node uses the filename_prefix %date:yyyy-MM-dd%/%date:hhmmss%_%KSampler.seed% to save the images on a folder with current date.

ControlNet sample output images

How to improve source images

• To prevent distortion, source images should have the same aspect ratio as the output image, or use the Crop and Resize resize_mode if you are happy with the preprocessor cropping the source image either vertically or horizontally.
• To extract poses, the subject should be properly centered. If the subject is too small/far away, the detected pose will be less precise, specially for hands and face detection. If the subject is too close, the generated images could result in a cropped subject.

Feedback

If you need help with this workflow, kindly leave a comment using the Disqus Comments section below.