How to assign each tile a class after tiling a big image and pass the tiles data directly into a trainer

[caitlin-cheng]

Hi lets say I have a huge image of high resolution and the content of the image is a repetitive pattern. There are some defective spots in the image and after tiling the image each defective spot is contained in a single tile. The rest tiles are good parts.

I'd like to tile the image use the tiler and use these tiles for patchcore and directly pass the good parts into the datamodule as the normal cases and the bad ones as abnormal cases.

Could you share some hints how I can achieve this without saving the tile images into separate folders.

[samet-akcay]

I believe this falls to @blaz-r's expertise :) @blaz-r can you provide your insights?

[blaz-r]

With the current tiler this would probably be a bit tricky. You could modify the tiler code here:

anomalib/src/anomalib/data/utils/tiler.py

Lines 250 to 273 in 6412fe1

	tiles = torch.zeros(
	(self.num_patches_h, self.num_patches_w, batch, channels, self.tile_size_h, self.tile_size_w),
	device=device,
	)
	# fill-in output tensor with spatial patches extracted from the image
	for (tile_i, tile_j), (loc_i, loc_j) in zip(
	product(range(self.num_patches_h), range(self.num_patches_w)),
	product(
	range(0, image_h - self.tile_size_h + 1, self.stride_h),
	range(0, image_w - self.tile_size_w + 1, self.stride_w),
	),
	strict=True,
	):
	tiles[tile_i, tile_j, :] = tensor[
	:,
	:,
	loc_i : (loc_i + self.tile_size_h),
	loc_j : (loc_j + self.tile_size_w),
	]
	# rearrange the tiles in order [tile_count * batch, channels, tile_height, tile_width]
	tiles = tiles.permute(2, 0, 1, 3, 4, 5)
	return tiles.contiguous().view(-1, channels, self.tile_size_h, self.tile_size_w)

to index out the parts that are anomalous around line 270. This would be a bit hacky to code due to way untiling works.

If the resolution is really large, that'd probably cause issues in any case, so I believe that splitting the image in advance would be easier.
With the upcoming tiled ensemble, this would be much easier but would still require some modifications to the code.

[caitlin-cheng]

Thank you both for your prompt replies. Saving the same data twice (full image or tiles) in my case is not ideal...I will try to work on the code. But it would be great if you could consider including this feature in the tiler in the future. Once a mask image of defects is provided, the tiles could then be easily classified accordingly.