Všechny publikace

ChunkyGAN: Real Image Inversion via Segments

  • DOI: 10.1007/978-3-031-20050-2_12
  • Odkaz: https://doi.org/10.1007/978-3-031-20050-2_12
  • Pracoviště: Katedra kybernetiky, Katedra počítačové grafiky a interakce, Skupina vizuálního rozpoznávání
  • Anotace:
    We present ChunkyGAN—a novel paradigm for modeling and editing images using generative adversarial networks. Unlike previous techniques seeking a global latent representation of the input image, our approach subdivides the input image into a set of smaller components (chunks) specified either manually or automatically using a pre-trained segmentation network. For each chunk, the latent code of a generative network is estimated locally with greater accuracy thanks to a smaller number of constraints. Moreover, during the optimization of latent codes, segmentation can further be refined to improve matching quality. This process enables high-quality projection of the original image with spatial disentanglement that previous methods would find challenging to achieve. To demonstrate the advantage of our approach, we evaluated it quantitatively and also qualitatively in various image editing scenarios that benefit from the higher reconstruction quality and local nature of the approach. Our method is flexible enough to manipulate even out-of-domain images that would be hard to reconstruct using global techniques.

STALP: Style Transfer With Auxiliary Limited Pairing

  • DOI: 10.1111/cgf.142655
  • Odkaz: https://doi.org/10.1111/cgf.142655
  • Pracoviště: Katedra počítačové grafiky a interakce
  • Anotace:
    We present an approach to example-based stylization of images that uses a single pair of a source image and its stylized counterpart. We demonstrate how to train an image translation network that can perform real-time semantically meaningful style transfer to a set of target images with similar content as the source image. A key added value of our approach is that it considers also consistency of target images during training. Although those have no stylized counterparts, we constrain the translation to keep the statistics of neural responses compatible with those extracted from the stylized source. In contrast to concurrent techniques that use a similar input, our approach better preserves important visual characteristics of the source style and can deliver temporally stable results without the need to explicitly handle temporal consistency. We demonstrate its practical utility on various applications including video stylization, style transfer to panoramas, faces, and 3D models.

Arbitrary style transfer using neurally-guided patch-based synthesis

  • DOI: 10.1016/j.cag.2020.01.002
  • Odkaz: https://doi.org/10.1016/j.cag.2020.01.002
  • Pracoviště: Katedra počítačové grafiky a interakce
  • Anotace:
    We present a new approach to example-based style transfer combining neural methods with patch-based synthesis to achieve compelling stylization quality even for high-resolution imagery. We take advantage of neural techniques to provide adequate stylization at the global level and use their output as a prior for subsequent patch-based synthesis at the detail level. Thanks to this combination, our method keeps the high frequencies of the original artistic media better, thereby dramatically increases the fidelity of the resulting stylized imagery. We show how to stylize extremely large images (e.g., 340 Mpix) without the need to run the synthesis at the pixel level, yet retaining the original high-frequency details. We demonstrate the power and generality of this approach on a novel stylization algorithm that delivers comparable visual quality to state-of-art neural style transfer while completely eschewing any purpose-trained stylization blocks and only using the response of a feature extractor as guidance for patch-based synthesis.

Interactive style transfer to live video streams

  • DOI: 10.1145/3407662.3407752
  • Odkaz: https://doi.org/10.1145/3407662.3407752
  • Pracoviště: Katedra počítačové grafiky a interakce
  • Anotace:
    Our tool allows artists to create living paintings or stylize a live video stream using their own artwork with minimal effort. While an artist is painting the image, our framework learns their artistic style on the fly and transfers it to the provided live video stream in real time.

Interactive Video Stylization Using Few-Shot Patch-Based Training

  • DOI: 10.1145/3386569.3392453
  • Odkaz: https://doi.org/10.1145/3386569.3392453
  • Pracoviště: Katedra počítačové grafiky a interakce
  • Anotace:
    We present a learning-based method to the keyframe-based video stylization that allows an artist to propagate the style from a few selected keyframes to the rest of the sequence. Its key advantage is that the resulting stylization is semantically meaningful, i.e., specific parts of moving objects are stylized according to the artist’s intention. In contrast to previous style transfer techniques, our approach does not require any lengthy pre-training process nor a large training dataset. We demonstrate how to train an appearance translation network from scratch using only a few stylized exemplars while implicitly preserving temporal consistency. This leads to a video stylization framework that supports real-time inference, parallel processing, and random access to an arbitrary output frame. It can also merge the content from multiple keyframes without the need to perform an explicit blending operation. We demonstrate its practical utility in various interactive scenarios, where the user paints over a selected keyframe and sees her style transferred to an existing recorded sequence or a live video stream.

Real-Time Patch-Based Stylization of Portraits Using Generative Adversarial Network

  • DOI: 10.2312/exp.20191074
  • Odkaz: https://doi.org/10.2312/exp.20191074
  • Pracoviště: Katedra počítačové grafiky a interakce
  • Anotace:
    We present a learning-based style transfer algorithm for human portraits which significantly outperforms current state-of-the-art in computational overhead while still maintaining comparable visual quality. We show how to design a conditional generative adversarial network capable to reproduce the output of Fišer et al.'s patch-based method that is slow to compute but can deliver state-of-the-art visual quality. Since the resulting end-to-end network can be evaluated quickly on current consumer GPUs, our solution enables first real-time high-quality style transfer to facial videos that runs at interactive frame rates. Moreover, in cases when the original algorithmic approach of Fišer et al. fails our network can provide a more visually pleasing result thanks to generalization. We demonstrate the practical utility of our approach on a variety of different styles and target subjects.

Za stránku zodpovídá: Ing. Mgr. Radovan Suk