# \[Pipelines\] Mask Generation Parameters

**URL:** <https://discuss.huggingface.co/t/pipelines-mask-generation-parameters/171394>\
**Category:** 🤗Transformers\
**Created:** [December 10, 2025, 6:43am UTC](https://discuss.huggingface.co/t/pipelines-mask-generation-parameters/171394 "2025-12-10T06:43:25Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![ChristianSi](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/christiansi/32/56418_2.png) [@ChristianSi](https://discuss.huggingface.co/u/ChristianSi)\
**Post date:** [December 10, 2025, 6:43am UTC](https://discuss.huggingface.co/t/pipelines-mask-generation-parameters/171394/1 "2025-12-10T06:43:25Z")

</div>

I noticed that the arguments you can pass to the **[call](https://github.com/huggingface/transformers/blob/cbc6716945cff1d8e124d344ba0150e6e27f8b6e/src/transformers/pipelines/mask_generation.py#L143)** method of `MaskGenerationPipeline `in `mask_generation.py` are similar but not identical to the parameters passed to the ` __init__ ` method of `SamAutomaticMaskGenerator`in the [official SAM GitHub Repo](https://github.com/facebookresearch/segment-anything/blob/dca509fe793f601edb92606367a655c15ac00fdf/segment_anything/automatic_mask_generator.py#L35). Some of the parameters like `box_nms_thresh`or `points_per_side`don’t exist in the HuggingFace repo.

I was wondering what’s the reason behind this omission. As far as I can tell these parameters are still used by SAM2 and it seems like the models that can be used for the mask generation task on HuggingFace are basically all SAM derivatives.

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [December 10, 2025, 8:26am UTC](https://discuss.huggingface.co/t/pipelines-mask-generation-parameters/171394/2 "2025-12-10T08:26:28Z")

</div>

The lack of parameters seems intentional.

* * *

The missing parameters are intentional.  
Hugging Face’s `MaskGenerationPipeline` is **not** a thin wrapper around Meta’s `SamAutomaticMaskGenerator` / `SAM2AutomaticMaskGenerator`. It is a separate automatic-mask-generation implementation with a different set of knobs, designed to work uniformly across multiple SAM-style models. Parameters that the HF implementation does not actually use, like `box_nms_thresh` and `points_per_side`, are not exposed.

I will unpack that in concrete terms.

* * *

## 1. What each implementation actually exposes

### Meta: `SamAutomaticMaskGenerator`

From the official SAM repo:

Constructor parameters include:

- Sampling:

- Crop schedule:

- Quality and stability:

- NMS:

- Small-region cleanup:

- Output format:

So Meta’s AMG is a fairly “expert” interface with distinct knobs for:

- grid density
- crop strategy
- per-crop and cross-crop NMS
- morphological cleanup
- and output encoding.

SAM2 and derivatives keep essentially the same parameter style (plus SAM2-specific flags such as `use_m2m`). For example, geospatial wrappers around SAM2 still configure `points_per_side`, `box_nms_thresh`, `crop_n_points_downscale_factor`, `min_mask_region_area`, and `use_m2m`. ([samgeo.gishub.org](https://samgeo.gishub.org/workshops/IPPN_2024/ "IPPN 2024 - segment-geospatial"))

### Hugging Face: `MaskGenerationPipeline. __call__ `

From the Transformers `mask_generation.py` you attached:

Parameters picked up by `_sanitize_parameters`:

- **Preprocess kwargs** (grid and crops):

- **Forward kwargs** (quality and cleanup):

- **Postprocess kwargs** :

Any other keyword argument, such as `points_per_side` or `box_nms_thresh`, is silently ignored by the pipeline because it is not added in `_sanitize_parameters`.

The HF tasks page for mask generation and the SAM/SAM2 docs show the same public surface: you configure `points_per_batch`, `points_per_crop`, `crops_n_layers`, `pred_iou_thresh`, `stability_score_thresh`, `crops_nms_thresh`, etc., not Meta’s full set. ([Hugging Face](https://huggingface.co/docs/transformers/en/tasks/mask_generation "Mask Generation"))

* * *

## 2. Concrete parameter mismatches

### 2.1 `points_per_side` vs `points_per_crop`

- Meta:

- HF:

So HF reparameterizes the sampling density:

- Same idea (uniform point grid over each crop), but:

- If you want equivalent density, you roughly set `points_per_crop ≈ points_per_side**2`, but the exact layout is controlled by the processor, not the pipeline.

### 2.2 `box_nms_thresh` + `crop_nms_thresh` vs `crops_nms_thresh`

- Meta:

- HF:

Because HF does not implement per-crop NMS, there is no meaningful place to plug in `box_nms_thresh`. Adding a parameter that the algorithm never uses would be confusing, so it is omitted instead.

### 2.3 `min_mask_region_area` vs `max_hole_area` / `max_sprinkle_area`

- Meta:

- HF:

So HF splits the single “minimum region area” concept into two more explicit morphological thresholds. The effect is similar (remove small artifacts), but it is not a direct 1:1 mapping, and the parameter name changes.

### 2.4 Other AMG-only arguments

Meta’s AMG and SAM2’s `SAM2AutomaticMaskGenerator` also expose parameters that have no stable meaning across all HF-supported mask-generation models:

- `point_grids`
- `output_mode` (binary vs RLE variants)
- SAM2-specific knobs such as `use_m2m`, `multimask_output`, etc. ([samgeo.gishub.org](https://samgeo.gishub.org/workshops/IPPN_2024/ "IPPN 2024 - segment-geospatial"))

HF’s `MaskGenerationPipeline` does not use these concepts at all. Instead it offers:

- `output_rle_mask` and `output_bboxes_mask` flags to optionally return extra outputs.
- A fixed internal representation for masks and bounding boxes.

Again, parameters that have no effect on the HF algorithm are not exposed.

* * *

## 3. Why Hugging Face omits those SAM/SAM2 parameters

There is no official HF comment that says “we deliberately removed `box_nms_thresh` and `points_per_side` because X”, but the design is clear from the code and docs.

Reasoning, point by point:

### 3.1 HF pipeline is a separate implementation, not a wrapper

- The HF task “mask generation” defines its own 3-stage pipeline:

- That logic lives in `MaskGenerationPipeline` and in the vision processors (`SamImageProcessor`, `Sam2ImageProcessorFast`). It is not calling Meta’s `SamAutomaticMaskGenerator` or `SAM2AutomaticMaskGenerator` anywhere. ([GitHub](https://github.com/huggingface/transformers/blob/main/src/transformers/pipelines/mask_generation.py "transformers/src/transformers/pipelines/mask\_generation. ..."))

Because it is an independent implementation, it:

- Keeps the general idea (grid of points, crops, thresholds, NMS, small region cleanup).
- But it is free to choose its own parameterization and internal steps.

### 3.2 Single API across multiple SAM-like models

HF wants one `mask-generation` task that works for:

- SAM v1 models (`facebook/sam-vit-*`).
- SAM-HQ, MedSAM, other fine-tuned variants.
- SAM2-based models. ([Hugging Face](https://huggingface.co/docs/transformers/en/model_doc/sam "SAM"))

To do that, they define a small set of operations that **all** supporting processors can implement:

- `generate_crop_boxes`
- `post_process_masks`
- `filter_masks`
- `post_process_for_mask_generation` ([GitHub](https://github.com/huggingface/transformers/blob/main/src/transformers/pipelines/mask_generation.py "transformers/src/transformers/pipelines/mask\_generation. ..."))

Then they expose only those parameters that make sense for every supported backend:

- Grid and crop density in terms of `points_per_crop`, `crops_n_layers`, etc.
- Quality thresholds (`pred_iou_thresh`, `stability_score_thresh`).
- One NMS threshold (`crops_nms_thresh`).
- Generic cleanup knobs (`max_hole_area`, `max_sprinkle_area`).

Things that are specific to one particular implementation (e.g. SAM’s per-crop NMS, SAM2’s `use_m2m`, custom point grids) would complicate that common API, so they are left out.

### 3.3 Pipeline philosophy: “few powerful knobs”

Hugging Face pipelines are designed as high-level, opinionated interfaces. The docs for the mask-generation task describe it as:

- “Automatic mask generation for images using `SamForMaskGeneration`.”
- With a small set of configuration options. ([Hugging Face](https://huggingface.co/docs/transformers/en/tasks/mask_generation "Mask Generation"))

This fits the general Transformers design:

- Pipelines expose a minimal parameter set.

- For advanced control, you drop down to:

So from HF’s perspective:

- Parameters like `points_per_batch`, `points_per_crop`, `crops_n_layers`, and the filtering thresholds cover 95% of what users need.
- Extra “power user” knobs from Meta’s AMG are better handled in custom code or directly in the SAM/SAM2 repos.

* * *

## 4. What this means for SAM2 and other derivatives

You are correct that SAM2’s own automatic mask generator still uses the full AMG-style set of parameters. For example, geospatial SAM2 wrappers show configuration like: ([samgeo.gishub.org](https://samgeo.gishub.org/workshops/IPPN_2024/ "IPPN 2024 - segment-geospatial"))

```python
sam2 = SamGeo2(
    model_id="sam2-hiera-large",
    points_per_side=64,
    points_per_batch=128,
    pred_iou_thresh=0.7,
    stability_score_thresh=0.92,
    stability_score_offset=0.7,
    crop_n_layers=1,
    box_nms_thresh=0.7,
    crop_n_points_downscale_factor=2,
    min_mask_region_area=25,
    use_m2m=True,
)

```

So:

- **Meta SAM2 code path** : still has `points_per_side`, `box_nms_thresh`, `min_mask_region_area`, `use_m2m`, etc.
- **HF `mask-generation` pipeline for SAM2** : uses the same _model weights_ but its own sampling and postprocessing pipeline, which ignores those SAM2-specific AMG knobs and uses HF’s parameter set instead. ([Hugging Face](https://huggingface.co/docs/transformers/en/model_doc/sam2 "SAM2"))

This is why you see the discrepancy:

- The models are SAM derivatives.
- The _automatic mask generator_ logic is HF’s own, not SAM2’s `SAM2AutomaticMaskGenerator`.

* * *

## 5. If you need the “missing” knobs

If you actually require per-crop NMS tuning or explicit `points_per_side`, there are only a few real options:

1. **Use Meta’s SAM/SAM2 repos directly**

2. **Use a wrapper that ports Meta’s AMG to HF checkpoints**

3. **Reimplement the AMG logic on top of HF models**

If you stay on `pipeline("mask-generation")`, you are constrained to HF’s reduced parameter surface by design.

* * *

## Summary

- The Hugging Face `MaskGenerationPipeline` re-implements automatic mask generation on top of `SamForMaskGeneration` and the SAM/SAM2 image processors. It does not call Meta’s `SamAutomaticMaskGenerator` or `SAM2AutomaticMaskGenerator`.

- Because the internal algorithm is different, some Meta AMG knobs have no direct counterpart:

- HF also wants one stable pipeline API that works for SAM, SAM-HQ, SAM2, and other derivatives, so it exposes only parameters that are meaningful across all of them. ([Hugging Face](https://huggingface.co/docs/transformers/en/tasks/mask_generation "Mask Generation"))

- SAM2 and many downstream projects still use the full AMG parameter set, including `points_per_side` and `box_nms_thresh`. If you need those, you must use Meta’s AMG implementation or a custom wrapper, not the HF `mask-generation` pipeline. ([samgeo.gishub.org](https://samgeo.gishub.org/workshops/IPPN_2024/ "IPPN 2024 - segment-geospatial"))

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/hellohellohello/original/2X/d/de4155eb4aa4108ecb32a1389d7cc37ae69f88b7.png) [@system](https://discuss.huggingface.co/u/system)\
**Post date:** [December 10, 2025, 8:26pm UTC](https://discuss.huggingface.co/t/pipelines-mask-generation-parameters/171394/3 "2025-12-10T20:26:37Z")

</div>

Closed on December 10, 2025, 8:26pm UTC

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.
