We present a patch-based harmonization network consisting of novel Patch-based normalization (PN) blocks and a feature extractor based on statistical color transfer. We evaluate our approach on available image harmonization datasets. Extensive experiments demonstrate the network's high generalization capability for different domains. Additionally, we collected a new dataset focused on portrait harmonization. Our network achieves state-of-the-art results on iHarmony4 and gains the best metrics on the synthetic portrait dataset.

For more information see our paper PHNet: Patch-based Normalization for Image Harmonization.

Clone and install required python packages:

git clone https://github.com/befozg/PHNet.git
cd PHNet
# Create virtual env by conda from env.yml file
conda create -f env.yml
conda activate phnet

# or install packages using pip
pip install -r requirements.txt

We present Flickr-Faces-HQ-Harmonization (FFHQH), a new dataset for portrait harmonization based on the FFHQ. It contains real images, foreground masks, and synthesized composites.

• Download link: FFHQH

Also, we provide some pre-trained models called PHNet for demo usage.

You can use downloaded trained models, otherwise select the baseline and parameters for training. To train the model, execute the following command:

python train.py

Refer to our config/train.yaml for training details.

To test the model, execute the following command:

python test.py

Refer to our config/test.yaml for inference details.

• Kapitanov Alexander
• Efremyan Karen
• Petrova Elizaveta
• Kaskov Evgeny

You can cite the paper using the following BibTeX entry:

@misc{efremyan2024phnet,
  title={PHNet: Patch-based Normalization for Portrait Harmonization}, 
  author={Karen Efremyan and Elizaveta Petrova and Evgeny Kaskov and Alexander Kapitanov},
  year={2024},
  eprint={2402.17561},
  archivePrefix={arXiv},
  primaryClass={cs.CV}}

• arXiv
• Github

This work is licensed under a variant of Creative Commons Attribution-ShareAlike 4.0 International License.

Please see the specific license.