We use a few cookies

Essential cookies keep the library working. With your permission, we'd also like to use analytics to see what's useful and improve the experience. Read our Privacy Policy or Cookie Policy.

Skip to main content
Home/e-Resources/Artificial Intelligence & Machine Learning/PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in Clustering
Conference ProceedingsArtificial Intelligence & Machine Learning

PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in Clustering

Checking your access to this record.

Abstract & Overview

We present a new framework for semantic segmentation without annotations via clustering. Off-the-shelf clustering methods are limited to curated, single-label, and objectcentric images yet real-world data are dominantly uncurated, multi-label, and scene-centric. We extend clustering from images to pixels and assign separate cluster membership to different instances within each image. However, solely relying on pixel-wise feature similarity fails to learn high-level semantic concepts and overfits to lowlevel visual cues. We propose a method to incorporate geometric consistency as an inductive bias to learn invariance and equivariance for photometric and geometric variations. With our novel learning objective, our framework can learn high-level semantic concepts. Our method, PiCIE (Pixel-level feature Clustering using Invariance and Equivariance), is the first method capable of segmenting both things and stuff categories without any hyperparameter tuning or task-specific pre-processing. Our method largely outperforms existing baselines on COCO [31] and Cityscapes [8] with +17.5 Acc. and +4.5 mIoU. We show that PiCIE gives a better initialization for standard supervised training. The code is available at https://github.com/janghyuncho/PiCIE.

Cite this record

Jang Hyun Cho, Utkarsh Mall, Kavita Bala, Bharath Hariharan (2021). PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in Clustering. arXiv (Cornell University). https://doi.org/10.1109/cvpr46437.2021.01652

Copy in another style

Cho, J. H., Mall, U., Bala, K., & Hariharan, B. (2021). PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in Clustering. arXiv (Cornell University). https://doi.org/10.1109/cvpr46437.2021.01652
Author names are used as deposited, so check their order before you submit.

Metadata is harvested from open-access indexes and can be incomplete. Check a citation against the published version before relying on it — see our open access & attribution policy.