In my knowledge, the input in transformer must be fixed resolution, in test time, often take patch overlap method to test image in transformer.in your code, I want to know how the method you take, and the idea. I saw that like any resolution can be feed in swinIR? how to do it? Looking forward your reply, thanku.!

hi,this is great work. I want to use this network for single image deraining, and what parts of this code can I modify? Or do you have any good suggestions? thanks!

在swinIR模型中，有img_size这个参数，例如为128， 在SwinLayer时，是input_resolution=（128， 128）, 比如我在测试的时候，我的输入图像不是(128, 128) 那么计算attention的时候 有一个判断， if self.input_resolution == x_size， else attn_windows = self.attn(x_windows, mask=self.calculate_mask(x_size).to(x.device))。我想请问一下如果图片大小不等于self.input_resolution=(128,128)时， 加入的参数 mask 这个是什么mask

Thank you for your work. I tried to train SwinIR but in my process of training swinir, I found that although the small swinir training is smooth, the loss often suddenly doubles when dim change to 180. Because of the memory problem, my batch_zie=16 lr=1e-4, may I have any special skills to let Is the training stable?

Hi, Jingyun, nice work! I just wonder why SwinIR function needs to set the 'img_size'. It is somehow kind of inconvenient, especially for test, since we usually want to test on different sizes of images, right? Is there any particular reason for this? Since Swin Transformer does not need this because they use padding operations. Besides, are there any requirements of the input size, i.e., must be the multiple of a number, or something else? Thanks.

In my opinion, the transformer costs much memory. And the paper pointed out that although swinir has fewer parameters, its speed is much slower than RCAN. So I'm curious about the cost of training, thank you.

Hi,

Thanks for sharing this interesting work. Table. 6, CBSD68, sigma=50 shows that IPT achieves 28.39 PSNR. However, the original paper of IPT shows that it can achieves 29.88 (in their Table. 2). Is there any difference with these two settings?

Thanks for releasing the code of SiwnIR, which is a really great work for low-level vision tasks.

However, when I train the SwinIR model with the guidance provided in the repo, I find the training efficiency is relatively low.

Specifically, the GPU utilization rate keeps 0 for a while from time to time (run 14 seconds and sleep 14 seconds). When the GPU utilization rate is 0, the CPU utilization is also 0. It's worth noting that I use the DDP training on 8 TITAN-RTX GPU cards with the default batch_size. I train the classic SR task with DIV2K dataset on X2 scale. After half-day training, The epoch, iteration and PSNR on Set5 are about 1500, 42000 and 35.73dB, respectively. So, it will takes about 5 days to finish the 500k iterations, far exceeding the 2 days reported in the README.

Could you please help me to figure out the reason for training efficiency?

Hi. I want to train the model with my own dataset. However, it keeps reporting RuntimeError: stack expects each tensor to be equal size, but got [3, 256, 256] at entry 0 and [3, 256, 252] at entry 1 Do I have any wrong setting? Thanks.

The part of json: "datasets": { "train": { "name": "train_dataset" // just name , "dataset_type": "sr" // "dncnn" | "dnpatch" | "fdncnn" | "ffdnet" | "sr" | "srmd" | "dpsr" | "plain" | "plainpatch" | "jpeg" , "dataroot_H": "HR" // path of H training dataset. DIV2K (800 training images) , "dataroot_L": "LR" // path of L training dataset

  , "H_size": 256                   // 96/144|192/384 | 128/192/256/512. LR patch size is set to 48 or 64 when compared with RCAN or RRDB.

  , "dataloader_shuffle": true
  , "dataloader_num_workers": 16
  , "dataloader_batch_size": 8      // batch size 1 | 16 | 32 | 48 | 64 | 128. Total batch size =4x8=32 in SwinIR
}
, "test": {
  "name": "test_dataset"            // just name
  , "dataset_type": "sr"         // "dncnn" | "dnpatch" | "fdncnn" | "ffdnet" | "sr" | "srmd" | "dpsr" | "plain" | "plainpatch" | "jpeg"
  , "dataroot_H": "testsets/Set5/HR"  // path of H testing dataset
  , "dataroot_L": "testsets/Set5/LR_bicubic/X4"              // path of L testing dataset

}

}

, "netG": { "net_type": "swinir" , "upscale": 4 // 2 | 3 | 4 | 8 , "in_chans": 3 , "img_size": 64 // For fair comparison, LR patch size is set to 48 or 64 when compared with RCAN or RRDB. , "window_size": 8
, "img_range": 1.0 , "depths": [6, 6, 6, 6, 6, 6] , "embed_dim": 180 , "num_heads": [6, 6, 6, 6, 6, 6] , "mlp_ratio": 2 , "upsampler": "pixelshuffle" // "pixelshuffle" | "pixelshuffledirect" | "nearest+conv" | null , "resi_connection": "1conv" // "1conv" | "3conv"

, "init_type": "default"

}

Instead of using pre-trained models, I trained the KAIR code and used the generated model for testing.

The KAIR training code produced 3 models:
40000_optimizerG.pth
40000_G.pth
40000_E.pth

Using these models, for testing the code in this repository, I am getting error:

(pytorch-gpu) C:\Users\Downloads\SwinIR-main>python main_test_swinir.py --task classical_sr --scale 2 --training_patch_size 48 --folder_lq testsets/Set5/LR_bicubic/X2 --folder_gt testsets/Set5/HR
loading model from model_zoo/swinir/40000_optimizerG.pth
Traceback (most recent call last):
  File "C:\Users\Downloads\SwinIR-main\main_test_swinir.py", line 253, in <module>
    main()
  File "C:\Users\Downloads\SwinIR-main\main_test_swinir.py", line 42, in main
    model = define_model(args)
  File "C:\Users\Downloads\SwinIR-main\main_test_swinir.py", line 174, in define_model
    model.load_state_dict(pretrained_model[param_key_g] if param_key_g in pretrained_model.keys() else pretrained_model, strict=True)
  File "C:\Users\anaconda3\envs\pytorch-gpu\lib\site-packages\torch\nn\modules\module.py", line 1406, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for SwinIR:
        Missing key(s) in state_dict: "conv_first.weight", "conv_first.bias", "patch_embed.norm.weight", "patch_embed.norm.bias", "layers.0.residual_group.blocks.0.norm1.weight", "layers.0.residual_group.blocks.0.norm1.bias", "layers.0.residual_group.blocks.0.attn.relative_position_bias_table",

40000_G.pth 40000_E.pth are testing fine

When I try to set pretrained models (003_realSR_BSRGAN_DFO_s64w8_SwinIR-M_x4_GAN.pth) paths in KAIR's train file;

, "path": { "root": "superresolution" // "denoising" | "superresolution" | "dejpeg" , "pretrained_netG": null // path of pretrained model , "pretrained_netD": null // path of pretrained model , "pretrained_netE": null // path of pretrained model }

it starts to train from scratch anyway. And when I copy from model_zoo right into /superresolution/swinir_sr_realworld_x4_gan/models/ thats not working either.

I finetuned the classical SwinIR model, then load the xxxx_E.pth for testing. I met that problem:

RuntimeError: Error(s) in loading state_dict for SwinIR: size mismatch for layers.0.residual_group.blocks.1.attn_mask: copying a param with shape torch.Size([4, 64, 64]) from checkpoint, the shape in current m odel is torch.Size([64, 64, 64]).

how to fix it?

Tried to open the PlayTorch demo on my iPhone XR (iOS 16.1) and got a "ViewPropTypes has been removed from React Native" error. PlayTorch v. 0.1.2.

Hi, I find that there are .png .jpg .bmp formats of training set pics, and I see that this may related to the windows size, so can you tell me if we should continue the window size as 8 or 7? And can we just put them into one single folder?

Thank you devs for great project. Unfortunately I had not success with converting to onnx and with trtexec converter from nvidia. Could you please, @JingyunLiang share tensorrt model? (realsr Large?) or someone else if succssed with conversion. Thank you very much in advance