GMS: Grid-based Motion Statistics for Fast, Ultra-robust Feature Correspondence

Abstract

Incorporating smoothness constraints into feature matching is known to enable ultra-robust matching. However, such formulations are both complex and slow, making them unsuitable for video applications. This paper proposes GMS (Grid-based Motion Statistics), a simple means of encapsulating motion smoothness as the statistical likelihood of a certain number of matches in a region. GMS enables translation of high match numbers into high match quality. This provides a real-time, ultra-robust correspondence system. Evaluation on videos, with low textures, blurs and wide-baselines show GMS consistently out-performs other real-time matchers and can achieve parity with more sophisticated, much slower techniques.

Publication

JiaWang Bian, Wen-Yan Lin, Yasuyuki Matsushita, Sai-Kit Yeung, Tan Dat Nguyen, Ming-Ming Cheng, GMS: Grid-based Motion Statistics for Fast, Ultra-robust Feature Correspondence, Conference on Computer Vision and Pattern Recognition (CVPR), 2017 [PDF][Project Page][Bib][C++][Youtube]

Core idea: Motion Statistics

Observation: True correspondences normally appear certain consistency in term of the motion while false correspondences do not. Hence, we predict that a true match may have many more supporting matches in its surrounding area than a false match.

 

Partionability: Assign the motion statistical value as a score for each feature correspondence, we discover that there exists very different numerical distribution between true and false correspondences in terms of such value.

 

 

Grid Framework

Grid Pattern: For fast computation, we assign the matches in the same cell with shared motion statistical value, rather than calculating motion statistics for each correspondence. With multi-cell generalization, our method also contains global information to some extent.

Rotation & Scale: The grid pattern can be rotated in 8 directions and also could be fitted with a large range of scale change by varying the size of the grid.

 

Sample Results

Wide Baseline: Qualitative wide-baseline matching results on Strecha Dense MVS datasets.

Repetitive Structure: Experiment results of our method on ZuBuD dataset.

Quantitative Evaluation

Matching Performance: Evaluation on the standard VGG and StrechaMVS datasets and the large-scale real-world TUM SLAM sequences.

Pose Estimation

Pose Estimation: Evaluation on all the datasets with the known camera poses. The success ratio and computation time are both evaluated.

Youtube Video Demo

 

 

 

9038total visits,38visits today

27 thoughts on “GMS: Grid-based Motion Statistics for Fast, Ultra-robust Feature Correspondence

  • May 8, 2017 at 4:34 pm
    Permalink

    你好,论文中的假设2和公式2一直没有理解,你能再稍微深入的解释一下吗?非常感谢!

    Reply
    • May 10, 2017 at 3:30 pm
      Permalink

      公式2说的是。如果一个匹配是错误的,那么它就有可能匹配到右边图像中任何一个特征点,因此它到达的区域是随机的。当然落在b区域中的可能性就是m/M.这里小m是b区域中特征点的个数,M是总共特征点的个数。

      Reply
      • May 19, 2017 at 10:44 am
        Permalink

        您的这篇文章外网打不开呢。另外,CVPR2017 给与会者发了论文集么?

        Reply
        • May 19, 2017 at 12:53 pm
          Permalink

          不清楚你为什么打不开,你可以发email给我,然后我给你发过去pdf.
          什么论文集?应该没有

          Reply
          • May 19, 2017 at 6:53 pm
            Permalink

            文章已经获得了,拜读大作ing。我在网上没找到CVPR2017的accepted paper list,找到的只有accepted paper id list,想看看里面专业相关的论文。因为有的会议会给与会者发一个U盘里面会有accepted paper合集,所以就问问CVPR是否也有^_^

  • May 23, 2017 at 10:39 am
    Permalink

    老师,您好!对于您论文中的哦你公式(3)不是太理解。如果已知a,b指向同一个区域,那么这个点匹配正确的概率不是变得大一些吗?为什么这两者是独立的?

    Reply
    • May 23, 2017 at 1:38 pm
      Permalink

      我们是首先假设所有匹配之间是相互独立的,然后才有了后面的分析。

      Reply
  • May 23, 2017 at 1:49 pm
    Permalink

    打不开你的论文,是否可以发到我邮箱?HonYan_Cao@163.com

    Reply
    • May 23, 2017 at 4:16 pm
      Permalink

      已发送

      Reply
  • May 23, 2017 at 9:02 pm
    Permalink

    您好,老师!
    拜读了您的论文和代码,简单有效啊。
    但对文中公式(14)的阈值近似不太理解,麻烦您解释指导下,非常感谢!

    Reply
    • May 23, 2017 at 9:51 pm
      Permalink

      非常抱歉,公式(14)上面一行有一处笔误。应该是 sf 而不是 sqrt(sf). 阈值这一块请参考代码,我之后会更正一下。

      Reply
      • May 24, 2017 at 10:03 am
        Permalink

        老师,
        您好!
        本人比较愚钝,还是不太理解sf近似等于sqrt(ni),还得麻烦您解释深入些,十分感谢啊!

        Reply
        • May 24, 2017 at 10:56 am
          Permalink

          sf在公式(8)中。 我们把pf(1-pf)当成一个常数提出去,所以Sf正比于sqrt(ni)。

          Reply
    • May 25, 2017 at 2:36 pm
      Permalink

      你好,能把您的邮箱发给我吗,关于运行代码有一些问题

      Reply
      • May 25, 2017 at 2:40 pm
        Permalink

        见Home。

        Reply
  • May 25, 2017 at 5:10 pm
    Permalink

    您好,运行您的代码时,发生错误。错误如下:
    /include/gms_matcher.h:60:2: error: too many initializers for ‘const int [8][9]’
    };
    ^

    请问可能原因是什么呢?

    Reply
    • May 25, 2017 at 9:54 pm
      Permalink

      那两个变量应该写在类外面。已经fix

      Reply
  • May 25, 2017 at 9:45 pm
    Permalink

    请问,你的这个gms算法可用于3d点云的特征提取和匹配吗?

    Reply
    • May 25, 2017 at 9:55 pm
      Permalink

      可以的。之前做过相应实验。

      Reply
  • May 26, 2017 at 9:48 am
    Permalink

    您好,老师您的matlab代码需要设置什么路径或者变量吗?我运行您的MATLAB代码,出现MexGMS.mexw64:找不到指定模块的错误,而MexGMS.mexw64文件我和m文件放在同一文件夹里了

    Reply
    • May 26, 2017 at 2:12 pm
      Permalink

      我刚更新了matlab的编译函数,你试一试。

      Reply
      • May 28, 2017 at 11:59 am
        Permalink

        好的,谢谢老师!

        Reply
    • May 27, 2017 at 10:29 pm
      Permalink

      您好,我出现跟你一样的错误,请问你问题解决了码?能否帮忙一下

      Reply
  • May 26, 2017 at 10:01 am
    Permalink

    论文下不下来,方便也不我一份,谢谢~

    Reply
    • May 26, 2017 at 2:14 pm
      Permalink

      已发送

      Reply
  • May 26, 2017 at 4:35 pm
    Permalink

    老师,您好,拜读了论文和代码,想法简单有效!但代码实现有一处不是很明白,有时间麻烦回答一下,谢谢

    int GetGridIndexLeft(const Point2f &pt, int type)函数的作用是将特征点的坐标映射到grid中吗?那样的话,下面程序实现是不是有错误?如果没错的话,这么做的目的是什么?
    x = floor(pt.x * mGridSizeLeft.width); //why? I think it should be “/” instead of “*”
    y = floor(pt.y * mGridSizeLeft.height);

    Reply
    • May 28, 2017 at 10:08 am
      Permalink

      pt已经被normalize到(0-1)。

      Reply

Leave a Reply

Your email address will not be published.