perceptor.losses

class perceptor.losses.AestheticVisualAssessment(aesthetic_target=10, mode='expected')

Bases: LossInterface

forward(images)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class perceptor.losses.BLIP(name='model_base_retrieval_flickr')

Bases: LossInterface

add_encodings_(encodings, weights=None)

add_images_(images, weights=None)

add_texts_(texts, weights=None)

property device

forward(images)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class perceptor.losses.CLIP(name='ViT-B-32', precision='fp32', jit=False)

Bases: LossInterface

add_encodings_(encodings, weights=None)

add_images_(images, weights=None)

add_text_off_(weight=None)

add_texts_(texts, weights=None)

property device

forward(images)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

mul_(multiplier)

training: bool

class perceptor.losses.CLOOB(name='16-epochs')

Bases: LossInterface

add_encodings_(encodings, weights=None)

add_images_(images, weights=None)

add_texts_(texts, weights=None)

property device

forward(images)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class perceptor.losses.LPIPS(name='squeeze', linear_layers=True, spatial=False)

Bases: LossInterface

forward(images_a, images_b)

Parameters

• images_a – images of shape (batch_size, 3, height, width) between 0 and 1

• images_b – images of shape (batch_size, 3, height, width) between 0 and 1

training: bool

class perceptor.losses.LiT(name='LiT-L16L', cache_dir=PosixPath('models'))

Bases: LossInterface

add_encodings_(encodings, weights=None)

add_images_(images, weights=None)

add_texts_(texts, weights=None)

property device

forward(images)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class perceptor.losses.Memorability

Bases: LossInterface

forward(images)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class perceptor.losses.OWLViT

Bases: LossInterface

add_encodings_(encodings: OWLViTEncodings, weights=None)

add_images_(images, weights=None)

add_texts_(texts: List[str], weights=None)

cpu()

Moves all model parameters and buffers to the CPU.

Note

This method modifies the module in-place.

Returns

self

Return type

Module

cuda()

Moves all model parameters and buffers to the GPU.

This also makes associated parameters and buffers different objects. So it should be called before constructing optimizer if the module will live on GPU while being optimized.

Note

This method modifies the module in-place.

Parameters

device (int, optional) – if specified, all parameters will be copied to that device

Returns

self

Return type

Module

property device

forward(images, top_k=5)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

to(device)

Moves and/or casts the parameters and buffers.

This can be called as

to(device=None, dtype=None, non_blocking=False)

to(dtype, non_blocking=False)

to(tensor, non_blocking=False)

to(memory_format=torch.channels_last)

Its signature is similar to torch.Tensor.to(), but only accepts floating point or complex dtypes. In addition, this method will only cast the floating point or complex parameters and buffers to dtype (if given). The integral parameters and buffers will be moved device, if that is given, but with dtypes unchanged. When non_blocking is set, it tries to convert/move asynchronously with respect to the host if possible, e.g., moving CPU Tensors with pinned memory to CUDA devices.

See below for examples.

Note

This method modifies the module in-place.

Parameters

• device (torch.device) – the desired device of the parameters and buffers in this module

• dtype (torch.dtype) – the desired floating point or complex dtype of the parameters and buffers in this module

• tensor (torch.Tensor) – Tensor whose dtype and device are the desired dtype and device for all parameters and buffers in this module

• memory_format (torch.memory_format) – the desired memory format for 4D parameters and buffers in this module (keyword only argument)

Returns

self

Return type

Module

Examples:

>>> linear = nn.Linear(2, 2)
>>> linear.weight
Parameter containing:
tensor([[ 0.1913, -0.3420],
        [-0.5113, -0.2325]])
>>> linear.to(torch.double)
Linear(in_features=2, out_features=2, bias=True)
>>> linear.weight
Parameter containing:
tensor([[ 0.1913, -0.3420],
        [-0.5113, -0.2325]], dtype=torch.float64)
>>> gpu1 = torch.device("cuda:1")
>>> linear.to(gpu1, dtype=torch.half, non_blocking=True)
Linear(in_features=2, out_features=2, bias=True)
>>> linear.weight
Parameter containing:
tensor([[ 0.1914, -0.3420],
        [-0.5112, -0.2324]], dtype=torch.float16, device='cuda:1')
>>> cpu = torch.device("cpu")
>>> linear.to(cpu)
Linear(in_features=2, out_features=2, bias=True)
>>> linear.weight
Parameter containing:
tensor([[ 0.1914, -0.3420],
        [-0.5112, -0.2324]], dtype=torch.float16)

>>> linear = nn.Linear(2, 2, bias=None).to(torch.cdouble)
>>> linear.weight
Parameter containing:
tensor([[ 0.3741+0.j,  0.2382+0.j],
        [ 0.5593+0.j, -0.4443+0.j]], dtype=torch.complex128)
>>> linear(torch.ones(3, 2, dtype=torch.cdouble))
tensor([[0.6122+0.j, 0.1150+0.j],
        [0.6122+0.j, 0.1150+0.j],
        [0.6122+0.j, 0.1150+0.j]], dtype=torch.complex128)

training: bool

class perceptor.losses.OpenCLIP(architecture='ViT-H-14', weights='laion2b_s32b_b79k')

Bases: LossInterface

add_encodings_(encodings, weights=None)

add_images_(images, weights=None)

add_texts_(texts, weights=None)

property device

forward(images)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class perceptor.losses.Resize(size=None)

Bases: LossInterface

forward(images_a, images_b, size=None)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class perceptor.losses.RuCLIP(name='ruclip-vit-base-patch32-224')

Bases: LossInterface

add_encodings_(encodings, weights=None)

add_images_(images, weights=None)

add_texts_(texts, weights=None)

property device

forward(images)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class perceptor.losses.SLIP(name='SLIP_CC12M')

Bases: LossInterface

add_encodings_(encodings, weights=None)

add_images_(images, weights=None)

add_texts_(texts, weights=None)

property device

forward(images)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class perceptor.losses.SimulacraAesthetic(model_name='ViT-L-14', aesthetic_target=10)

Bases: LossInterface

forward(images)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class perceptor.losses.Smoothness

Bases: LossInterface

forward(images)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class perceptor.losses.SphericalDistance(model)

Bases: LossInterface

forward(images_a, images_b)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class perceptor.losses.StyleTransfer(style_images=None)

Bases: LossInterface

encode(images)

forward(images_a, images_b=None)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

loss(encodings_a, encodings_b)

training: bool

class perceptor.losses.SuperResolution(name='x2', pre_downscale=None, half=True, mode='bicubic')

Bases: LossInterface

forward(images)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class perceptor.losses.SuperResolutionDiscriminator(name='RealESRGAN_x4plus_netD')

Bases: LossInterface

forward(images)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

training: bool

class perceptor.losses.VelocityDiffusion(model, noise, from_ts=0.5, resample_ts=0.3)

Bases: LossInterface

compensate_noise_(from_denoised, to_denoised)

diffuse_denoise(denoised, **extra_kwargs)

forward(images, frozen_diffused_denoised)

Defines the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

guided_resample_(denoised, guidance_scale=0.5, clamp_value=1e-06, **extra_kwargs)

Resamples noise in direction of the gradient

Usage:

with diffusion.guided_resample_(images) as diffused_denoised:

clip(diffused_denoised).backward()

noise_step_(from_denoised, from_t, to_t, to_denoised)

1. Step to to_t

2. Add noise to from_t

training: bool