CN104603833A - A method and system for linking printed objects with electronic content - Google Patents
A method and system for linking printed objects with electronic content Download PDFInfo
- Publication number
- CN104603833A CN104603833A CN201380040688.XA CN201380040688A CN104603833A CN 104603833 A CN104603833 A CN 104603833A CN 201380040688 A CN201380040688 A CN 201380040688A CN 104603833 A CN104603833 A CN 104603833A
- Authority
- CN
- China
- Prior art keywords
- image
- text
- array
- digital
- word
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/40—Document-oriented image-based pattern recognition
- G06V30/41—Analysis of document content
- G06V30/413—Classification of content, e.g. text, photographs or tables
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/955—Retrieval from the web using information identifiers, e.g. uniform resource locators [URL]
- G06F16/9554—Retrieval from the web using information identifiers, e.g. uniform resource locators [URL] by using bar codes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/56—Information retrieval; Database structures therefor; File system structures therefor of still image data having vectorial format
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/80—Information retrieval; Database structures therefor; File system structures therefor of semi-structured data, e.g. markup language structured data such as SGML, XML or HTML
- G06F16/84—Mapping; Conversion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/40—Document-oriented image-based pattern recognition
- G06V30/41—Analysis of document content
- G06V30/414—Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30176—Document
Abstract
The present invention relates to a system and a method for linking printed objects to electronic content. The system of the present invention enables users of an MMD (mobile media device) to capture an image of a printed object (e.g., the object can be at least a portion of a printed article) and to get in return a corresponding electronic content (e.g., a video clip, picture, text, link to a web page and the like). Examples of the invention provide an image processing method in which the captured image is converted into an array of clustered vectors (refer herein as descriptors) that is based on the inter-line spacing and interword spacing of text and non-text objects, at samples across the area of the image. The method further comprises prior to the sending of the array of binary vectors, converting each vector from said array into a unique charters string ('hashwords'). In another aspect the present invention relates to a computer program comprising computer program code means adapted to perform the method for linking printed objects to electronic content when said computer program is run on a computer based device, such as an MMD or a server.
Description
Technical field
The present invention relates to media services field.More specifically, the present invention relates to the method and system for providing the ocular connection between print media and Digital Media.
Background technology
In recent years, the mobile device of such as mobile phone becomes more and more general.A new generation's mobile device (such as, smart mobile phone and panel computer) provides the camera specification of enhancing, high speed internet is connected and rapid internal processor.
For carry out between printing material with digital content the example of the solution linked be use bar code.But the size and appearance due to the such as bar code of QR code does not allow it to have a lot of example in the page of text of printing, so it stands serious intrinsic problem, hyperlink usually can be like this.At a page, there is the layout of multiple bar code for the page and require unacceptable change.
The function that the object of the invention is to use these to strengthen can become intermediary between printing material and e-file and connector via internet to make such Senior mobile device.
An object of the present invention is to provide comfortable print media to read and the updatability of online Media and rich to user.
Another object of the present invention allows in print media when not disturbing the look and feel of printing object multiple " off-line hyperlink ".
Along with description is carried out, other objects of the present invention and advantage will become obvious.
Summary of the invention
The present invention relates to a kind of method for printing object being linked to digital content, comprising:
A) provide at least one digital copies of printing object to the database of server, wherein, described digital copies represents described printing object at least partially, and wherein, described digital copies is identical with the contents and distribution of described printing object;
B) by extract with the coordinate of each link of preserving in described digital copies or via external linkage list provide described digital copies layout at least partially with the associating of digital content;
C) process to distinguish text object and this two class object of non-text object to described digital copies, wherein, each text object represent word at least partially or at least two or more the adjacent combinatorics on words gone up in any direction relative to the plane of described printing object;
D) relative to the reference position of the coordinate of definition, each object in two class objects is encoded in the layout of described printing object with it according to the visual shape of each object in two class objects, thus described digital copies is expressed as the vector array of binary format;
E) create the cluster of similar binary vector and distribute random character array to each described cluster, and described character array is saved as the encoder dictionary of " Hash word (hashword) ", thus described binary vector is converted to the textual form that can search for;
F) image at least partially of described printing object is caught by mobile device;
G) object distinguishing text object and Graph-Oriented picture is processed to described image of catching, and the visual shape according to each described object is encoded to each described object relative to the reference position of the layout of described printing object with it, thus is the vector array of binary format by described digital image representation of catching;
H) via communication network, described binary vector array is sent to described server;
I) each Clustering Vector is converted to Hash word;
J) execution contexts search with identify substantially to the corresponding Hash word of the Hash word optimum matching of encoding according to digital copies, wherein, before transmitting in the mobile device local or remotely convert described array to Hash word at described server place after sending;
K) to representing that the vector array of the image of catching performs the geometric match in the middle of optimum matching;
L) calculate position on optimum matching digital copies of the part of catching and extract the specific link being intended to caught image; And
M) digital content of association or its respective links are sent to described mobile device from described server.
According to embodiments of the invention, the process of digital picture comprises: detect lines spacing and word space on the region of described digital picture, and word boundary is also determined in the text filed and non-textual region detected in often going, thus determines each text object and each non-text object.
According to embodiments of the invention, this process also comprises: by carrying out pre-service to the numeral application image Processing Algorithm of described image to caught image.This pre-service comprises: perform different manipulated images and conversion correction, such as correct the image rotation of (skew correction) for skew.
According to embodiments of the invention, the method also comprises: before transmission binary vector array, each vector in described array is converted to unique character string (" Hash word ").
On the other hand, the present invention relates to a kind of system being suitable for the method performed for printing object being linked to digital content.
On the other hand, the present invention relates to a kind of computer program comprising computer program code instrument, when described computer program runs on computer based device (such as MMD or server), computer program code instrument is suitable for performing the method for printing object being linked to digital content.
Accompanying drawing explanation
In the accompanying drawings:
Fig. 1 generally show the block diagram of system according to an embodiment of the invention for printing object and digital content being linked;
Fig. 2 is the process flow diagram that generally show descriptor leaching process according to an embodiment of the invention;
Fig. 3 is the process flow diagram that generally show method of the present invention; And
Fig. 4 generally show and according to an embodiment of the invention image is divided into text chunk and non-textual section so that the example to each section of application algorithms of different.
Embodiment
Run through this instructions, term " mobile media device " or (MMD), for representing computer based device, include but not limited to mobile phone, flat computer, notebook and comprise any other electronic installation of camera or other image capture apparatus.
Now with reference to some embodiments of the present invention, its example shown in the drawings.Applicable similar or same reference numerals can be used Anywhere in the accompanying drawings, and its can representation class like or identical function.Only for purpose of explanation, accompanying drawing depicts embodiments of the invention.Those skilled in the art will easily recognize according to description below, when not deviating from principle of the present invention described herein, can adopt shown structure and the alternative embodiment of method herein.
System of the present invention makes the user of MMD can catch printing object (such as, this object can be printing item at least partially) image and obtain corresponding digital content (such as, video segment, picture, text, link etc. to webpage) as return.
Example of the present invention provides following image processing method: convert caught image to Clustering Vector array (being called descriptor herein) with the sample on image-region, and this Clustering Vector array is based on the line space of text object and non-text object and word space.
To be described each aspect of the present invention and exemplary computing operation environment.Fig. 1 and discussion are below intended to provide the concise and to the point whole description to realizing applicable computing environment of the present invention wherein.Although under the conventional background of the program module that the application program run in the operating system being combined in MMD performs, present invention is described, person of skill in the art will appreciate that, can also realize the present invention in conjunction with other program modules.
Fig. 1 diagrammatically illustrates the system 10 according to an embodiment of the invention for printing object and digital content being linked.This system comprises MMD 150 and server database 130.MMD 150 is provided with the MMD specified and applies (such as, client), its be suitable for catching source object 100 image in case obtain as return with source object 100 catch digital content that image is associated 120 (such as, video segment, picture, text, link etc. to webpage).Source object 100 can be any printing material, such as book, magazine, pamphlet etc.
The digital copies 110 (such as, pdf document form) with source object 100 with the source object 100 of identical topology is stored in server 130.In other words, source object 100 is printed copies of stored digital copies 110.Digital copies 110 can comprise single image or multiple image, and wherein, such as, each image can represent at least one page (or sometimes representing the only part of at least one page) of the printing item of such as book or magazine.The link of associated electrical content 120 to source object 100 can be pre-defined in linked list 110A.Linked list 110A comprise link object (word, sentence, picture etc.) and under the URL of digital content its coordinate in the page; Or can embed in the digital copies 110 of source object 100 as hyperlink.
Such as, one or more hyperlink can embed (such as, with the form of PDF) in the digital copies 110 provided by the publication side 160 of digital copies 110.According to this example, publication side 160 can send/upload digital copy 110 (such as, the image of the source page and linked list 110A) or there is the pdf document of final layout of the embedded hyperlink to server 130.
To digital copies 110 and linked list 110A process with the correlated characteristic of extraction source object 100.This process comprises: a. has the lists of links 110A of its coordinate, as in fact appeared in source object 100 from pdf document extraction link/hyperlink or preservation; And b. converts digital copies 110 to descriptor, then convert text string (that is, Hash word) to.As will be described in more detail hereinafter, this will make it possible to preserve the less file of size and search for faster afterwards and compare process.
In MMD side, after user utilizes MMD 150 source object 100 takes pictures to printing, the MMD application of specifying processes obtained image, and will represent that the data of result are sent to server 130.The MMD application of specifying comprises the encryption algorithm 140A obtained image being converted to coding form (descriptor).According to embodiments of the invention, the coded image that the image conversion of catching becomes to be defined by multiple Clustering Vector by this process, wherein, the information of the part in the photo captured by each vector representation.With reference to Fig. 2 and Fig. 3, the method/algorithm being used for generating coded image will be described in more detail and also hereinafter.
At server side, system 10 uses encryption algorithm 140A to convert the page layout of digital copies to descriptor, and use searching algorithm 140B (being similar to the process undertaken by MMD 150 pairs of source objects 100), searching algorithm 140B the source object 100 at database 130 place of server conventional images (namely, digital copies 110) carry out identifying and mating with between the coded image sent from MMD 150, coded image can comprise with only some similaritys of source object 100 (such as, an only part for image, distortion, dimness, does not focus).
Extract link from digital copies 110 by additional advantage provided by the invention, wherein, each position (coordinate of each link such as, in the page) linked that is in digital copies 110 form of system 10 extraction source object 100 or that be associated with this digital copies 110 form.When finding the digital copies of source object 100 and mating between the data sent from MMD 150, any focus (that is, coordinate) occurs, this is because data can be mated with corresponding linking, then, relevant digital content or the link to it can be sent to MMD 150.Such as, MMD 150 may be used for illustrating/show digital content to user.
According to embodiments of the invention, system 10 is provided with network account ability, thus allows user to open personal user's network account 160.Such as, the link to digital content is also stored in personal user's network account 160 for using (such as, reset, share, comment etc.) later.
Forward Image Coding Algorithms to now, wherein, according to the present invention, the image procossing of digital picture is used for two class objects in differentiate between images: text object and non-text object (such as, picture).Usually, identical process is applicable to the image that MMD 150 catches and the digital copies be stored in server 130.But, due to its attribute (namely, camera angle, illumination condition, distortion etc. due to MMD), the image of catching required the usual unwanted preprocessing tasks of some digital copies usually before coded treatment, such as required the aligning of the image of catching due to the camera angle of MMD.
But, for brevity, be described to the process and example that are found the Image Coding obtaining best result hereinafter.Although certainly can use the algorithm of alternative to obtain more coarse result, in addition, can adopt the algorithm suitable with algorithm described herein, the present invention comprises all above-mentioned algorithms.
MMD 150 can comprise the following steps the process of the image of catching:
-by the image that should be used for catching source object 100 (such as, printing item) of specifying of MMD 150.This appointment can comprise visual target frame (touch-screen such as, being presented at smart mobile phone makes camera be intended to catch image simultaneously) to help user to take pictures to area-of-interest relative to the whole layout of the page;
-image of catching is processed, this can comprise the such as common preprocessing tasks such as gradation conversion, image aligning and converted to by the image of catching and utilize the vector representation of descriptor array (that is, converting coded image to).This conversion comprises and uses different algorithm from the image zooming-out text descriptor of catching and non-textual descriptor and carry out cluster to extracted descriptor; And
-coded image is sent to the digital content that server 130 is associated with the image searched with catch.
At server 130 place, use same procedure and dictionary as the conversion of the descriptor of digital copies, convert the descriptor cluster received from MMD 150 to Hash word.Then, carry out searching for searching and the relevant portion the digital copies of the obtained images match sent from MMD, then search and the digital content of catching image object and being associated.Usually, search procedure can comprise the following steps:
-the Hash word of the image of catching is mated with the Hash word of stored digital copies;
-utilizing the application of MMD to search, its coordinate mates with the target frame being used as target by user links;
-be associated to relevant digital content; And
-digital content or its link are sent to MMD 150.Such as, at MMD 150 place, user can play digital content, and shared electron content or store electrons content are for later use.
According to some embodiments of the present invention, catch Image Coding process and can comprise the following steps, as shown in Fig. 2 generally:
-catch image (step 20) from printing item;
-carrying out pre-service (step 21) to catching image, this can comprise usual task, such as gradation conversion, image aligning etc.;
-application dividing processing is to separate (step 22) by the text element in image and non-text elements; And
-carry out text and non-textual descriptor extraction (step 23 and step 24), to convert each text object in the image of catching to vector representation.
Descriptor extraction process can comprise the following steps:
A. descriptor extraction-text descriptor
1., if needed, the image of catching is transformed into gray level;
2. pair image applications deskew (deskew) algorithm of catching is with the line of text horizontal aligument will occurred in the image of catching.This algorithm can be any deskew algorithm or equivalent algorithm, the PD deskew algorithm such as provided by Leptonica image procossing storehouse;
3. the image applications image binaryzation (such as, by using the Sauvola method being used for local binarization) of pair deskew;
4. search institute's spottiness (blob) in binary image, wherein, for the text object in image, spot can represent a part for word or word, and for non-text object, spot can represent part or other image objects of figure.Based on the line space in the image of catching and word space, obtain representing the array of spots of the visible deployment of the word in the image of catching.Spot can refer to and a kind ofly comprise bounding box that is bending or asymmetric profile;
5., for each spot centers, obtain the distance of pixel to spot centers hithermost with it;
6. calculate the intermediate value of the distance obtained;
7. be that the rectangle kernel of f* intermediate value (such as, f=2) carrys out applied morphology computing (morphological operation) by approaching size, wherein, f is the adjustable parameter of this application.Term " approaches " computing in this article referring to morphology noise remove, therefore, approaches operation and enough close spot is mixed together.
8. after approaching computing, again search the institute's spottiness in image.If have enough text (such as, even 3 these generals of style of writing it is enough) in the picture, then all words will in single spot;
9., according to the adjustable parameter of this algorithm, little spot and very large spot are filtered;
10. calculate to be used as non-text object (such as, scheme and image) and text filed between the bounding box of each spot of mask (mask);
11. obtain scale-of-two deskew image (that is, binary image), and for each bounding box, calculate the barycenter of " on (the connection) " pixel occurred wherein;
12. for each bounding box: the mean distance (wherein, x=fwidth and y=fheight) searching wherein " on " pixel (in x dimension and y dimension) according to calculated barycenter.In binary representation, " 1 " refers to black picture element (that is, " on " pixel), and " 0 " refers to white pixel (that is, without pixel);
13. for each bounding box: utilize width f
x* fwidth and height f
y* fheight calculates the new bounding box centered by calculated barycenter, wherein, and f
xand f
yit is the adjustable parameter of this algorithm;
14. utilize n
yn in row and often row
xeach new bounding box bilinearity is scaled the image of fixed size by individual pixel; Wherein, n
yand n
xit is the adjustable parameter of this algorithm; And
Zoomed image flattens as vector by 15. line by line, and wherein, the zoomed image of described leveling refers to text descriptor in this article.
B. descriptor extraction-non-textual describes:
1. adjust the size (as obtained in the steps A .2 above) (identical with steps A .7 above) of the grayscale image of deskew according to zoom factor s* intermediate value, wherein, s is the adjustable parameter of this application;
2. relative to the image after adjust size to above-mentioned steps A.10 in the mask that calculates carry out convergent-divergent, and to the mask after the image applications convergent-divergent after adjust size;
3. the Corner Detection Algorithm by applying such as AGAST Corner Detection device extracts point of interest, and if point of interest fall into text filed inside, then abandon point-of-interest; And
4. for each effective unique point, scale-of-two descriptor is extracted by key application point descriptor algorithm (such as FREAK: retina key point algorithm fast), wherein, this scale-of-two descriptor table diagram picture (non-textual) descriptor.
Give an only example of each in dissimilar image procossing.It will be apparent to one skilled in the art that much different image processing techniquess can be used for the pattern in detected image, certainly, text is only picture pattern.Thus to there is much different which parts for detected image be text object and which part is the process of non-text object.
According to embodiments of the invention, in order to be provided for searching the fast search process of mating between image and the digital copies of stored source object of catching, by extract from digital copies, descriptor in the server 130 of system converts Hash word (illustration and text juxtaposed setting this) form to.Hash word is stored in encoder dictionary, and makes it possible to the database (by utilizing common search engine, such as google search engine, SQL etc.) of search server 130 in the form of text.For each text descriptor and non-textual descriptor, dictionary building process can comprise the following steps:
1. the database (such as, as the pdf document form with digital copies provides) of Given Graph picture, extracts all text descriptor from all literal copy; And
2. the cluster analysis by applying such as layering k-mean algorithm for binary vector carries out cluster to all descriptors;
3. for each cluster distributes random Hash word (such as, 6 alphabetical word); And
4., for non-textual descriptor, repeat the identical process of step 1 to step 3.
Referring now to Fig. 3, be described to the process flow diagram mainly illustrated for realizing the embodiment being used for the method for the present invention printing object and digital content linked now.The method of this embodiment may include following steps:
-obtain image (step 30) by MMD 150;
-(step 31) is extracted for the text object in obtained image and non-text object performance description symbol, and descriptor is sent to the server 130 of system from MMD 150;
-use above-mentioned dictionary to convert each received descriptor to text Hash word format (step 32); The conversion of descriptor to Hash word is hereafter being described in more detail;
-utilize the descriptor changed to carry out text search (step 33) to the Hash word appeared in dictionary.As above noted earlier, before catching image by MMD 150, be extracted in from digital copies (such as digital copies 110) the Hash word (such leaching process is briefly represented by Reference numeral 131 in the drawings) occurred dictionary;
-search the best matching result (step 34) of carried out text search;
-for each optimum matching, perform geometric match to provide the homography score (homograph score) (step 35) of the optimum matching to descriptor.Geometric match refer to represent the descriptor of image that obtains with between the descriptor representing each digital copies to the task that the geometric position of descriptor is mated;
-in the middle of optimum matching, select optimum (that is, the highest homography score-as described in more detail below) (step 36) according to the maximum matching number amount of the descriptor between obtained image and digital copies; And
-for selected optimum, the associated link of the closest picture centre obtained in location or digital content (step 37), and send it to MMD 150 (step 38).
Descriptor is to the transfer process of Hash word:
Random character array (such as, alphabetical) is distributed for the given cluster as the similar descriptor of the descriptor described in above and dictionary creation process-create and for cluster.These character group will be saved the encoder dictionary into " Hash word ".
The homography score of database result
Given query image and descriptor thereof are (namely, as from extract in the image that obtains-source object) and possible target image (namely, the target image of digital copies) and descriptor, use algorithm that be applicable in the open CV storehouse of well-known PD, that be called as RANSAC homography to search the descriptor of the maximum quantity of the homography between applicable two images.Wherein, assuming that the most likely suitable document of the document (that is, digital copies) with highest ranking.
According to embodiments of the invention, mark source object as follows: user will know, this object is supported by technology of the present invention, such as, by adding visual indicators to source object.
According to embodiments of the invention, MMD also can be made to apply can select will to a part for the text object of its application image process or non-text object.This can by display-object element, operate the application of MMD to catch image to carry out from printing object simultaneously.Such as, as above-mentioned above, object element can be the form of target frame.
Technician will be understood that, layout described in figure produces following system: it allows object (showpiece in such as print page, symbol, billboard, exhibition or terrestrial reference) to link to digital content (vision of such as, advertisement, company's site, representative of sales & marketing, text message, reward voucher or any other kind and/or audio content).
System of the present invention makes it possible in any printed matter (such as, word, sentence, picture, advertisement etc.), add multiple link, when not trading off to the look and feel of printed matter the spitting image of the function of the hyperlink in webpage.In addition, system of the present invention provides the ability of the existing printed article of link (not under reader adds tagged situation).In this case, this system only can identify print page.Such as, if there are the multiple links relating to print page, then this system will show multiple button for reader from wherein selecting in the application of MMD.Therefore, the object that such as secondhand book a lot of have printed and new printing object are provided with the function being similar to hyperlink.It allows to be used in the page many links that may exist, and just as in the online page, but does not trade off to the look and feel.
Additional advantage provided by the invention is ability video content (commercial advertisement such as, on TV) being linked to digital content.Use the present invention, user can use mobile phone should be used for the picture/fragment of the program that televising is play, and acquires the digital content (such as, reward voucher, website, marketing information, directly buy) of mobile phone.Such as, in this case, the server of system is by store video fragment, and caught frame mates with the frame in fragment by this system.
Although combined the iamge description of being caught by MMD camera the method above, but, can to the image applications image procossing provided by other image capture apparatus.
Said method can be embodied as the computer program performing image processing function.Thus, can comprise for realizing equipment of the present invention the computing machine processing to perform graphical analysis and follow-up correction to digital image file.Usually, program module comprises routine, program, assembly, data structure and performs particular task or realize the structure of other types of particular abstract data type.Also put into practice the present invention in a distributed computing environment, in a distributed computing environment, executed the task by the remote processing device of communication network links.In a distributed computing environment, program module can be arranged in local memory storage device and remote memory storage device.
The function above described can by being stored in computer-readable medium and the executable code run in the system based on one or more processor and instruction perform.But, also can using state machine and/or hard-wired electronic circuit.In addition, about example process described herein, do not need to reach all treatment states, not necessarily perform these states with the order illustrated yet.In addition, some treatment state being illustrated as performing serially can be performed concurrently.
All foregoings can be understood better by illustrative and nonrestrictive example below.Example that described and described herein occurs and term is intended to be illustrative and exemplary, but not limits the scope of the present invention for required protection by any way.
Fig. 4 shows the example digital image of source object according to an embodiment of the invention, and it represents the printing layout of the page of the print newspapers different processing stage.The original digital image of print newspapers is represented (such as, being obtained by MMD) by Reference numeral 41.At next processing stage, show the binary representation (as shown in by Reference numeral 42) of the digital picture of source object before extracting text descriptor and non-textual descriptor, after application partitioning algorithm.At next processing stage, each text object representing (as shown in by Reference numeral 43) at least partially or relative to the bounding box form of at least two or more adjacent printing combinatorics on words that the plane of printing layout goes up in any direction by printed words.At next processing stage, set-up dirgram picture is to extract text descriptor and non-textual descriptor, and wherein, Reference numeral 44 represents the digital picture that text object is only shown, and Reference numeral 45 represents the digital picture that non-text object is only shown.Reference numeral 45 represents at application partitioning algorithm and covers text, prepares non-textual descriptor with the example of the digital picture of the source object after extracting.Reference numeral 44 represent to text object application partitioning algorithm, prepare text descriptor with the example of the digital picture of the source object after extracting.
In this embodiment, each text object is converted into quadrangular forms or bounding box substantially, and comprise word generation corresponding higher form, such as alphabetical b, d, f, g, h, k, l, p, q, t and the y of the letter from the common virtual alignment line expansion of often going.According to font attribute, each different word (or according to line space and word space, a part for word or two or more combinatorics on words) can unique forms be produced.
All foregoing descriptions and example provide for purposes of illustration, and are not intended to limit the present invention by any way.When being all no more than scope of the present invention, much different mechanism, analytical approach, electronics and logic element can be adopted.
Claims (6)
1., for printing object being linked to a method for digital content, comprising:
A) provide at least one digital copies of printing object to the database of server, wherein, described digital copies represents the printing layout of described printing object at least partially;
B) by extract and preserve each link in described digital copies coordinate or via external linkage list by described digital copies being associated with digital content at least partially in layout;
C) process to distinguish text object and this two class object of non-text object to described digital copies, wherein, each text object represent word at least partially or at least two or more the adjacent combinatorics on words gone up in any direction relative to the plane of described printing layout;
D) relative to the reference position of the coordinate of definition, each object in this two class object is encoded in described printing layout with it according to the visual shape of each object in two class objects, thus described digital copies is expressed as the vector array of binary format;
E) create the cluster of similar binary vector and distribute random character array to each described cluster, and described character array is saved as the encoder dictionary of " Hash word ", thus described binary vector being converted to the textual form that can search for;
F) image at least partially of described printing object is caught by mobile device;
G) process to distinguish text object and non-legible object to described image of catching, and the visual shape according to each described object is encoded to each described object relative to the reference position of described printing layout with it, thus is the vector array of binary format by described digital image representation of catching;
H) via communication network, described binary vector array is sent to described server;
I) each Clustering Vector in Hash word is changed and execution contexts search with identify substantially to represent described in the corresponding Hash word of the Hash word optimum matching of image of catching, wherein, before described transmission in described mobile device local or remotely convert described array to Hash word at described server place after described transmission;
J) to representing that the described vector array of the image of catching performs the geometric match in the middle of optimum matching;
K) calculate position on optimum matching digital copies of the part of catching and extract the specific link being intended to caught image; And
L) digital content of association or its respective links are sent to described mobile device from described server.
2. method according to claim 1, wherein, the process of described digital picture comprises: detect lines spacing and word space on the region of described digital picture, and word boundary is also determined in the text filed and non-textual region detected in often going, thus determines each text object and each non-text object.
3. method according to claim 1, wherein, described process also comprises: by carrying out pre-service to the numeral application image Processing Algorithm of described image to caught image.
4. method according to claim 1, wherein, described pre-service comprises: perform the image rotation being used for skew and correcting.
5. method according to claim 1, wherein, also comprises: before the described binary vector array of transmission, convert each vector in described array to unique character string.
6. comprise a computer program for computer program code instrument, when described computer program runs on computer based device, described computer program code instrument is suitable at least step c and the steps d that enforcement of rights requires 1.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US201261681150P | 2012-08-09 | 2012-08-09 | |
| US61/681,150 | 2012-08-09 | ||
| PCT/IL2013/050679 WO2014024197A1 (en) | 2012-08-09 | 2013-08-08 | A method and system for linking printed objects with electronic content |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104603833A true CN104603833A (en) | 2015-05-06 |
| CN104603833B CN104603833B (en) | 2018-12-14 |
Family
ID=50067498
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201380040688.XA Expired - Fee Related CN104603833B (en) | 2012-08-09 | 2013-08-08 | Method and system for linking printing object with digital content |
Country Status (5)
| Country | Link |
|---|---|
| US (1) | US9916499B2 (en) |
| EP (1) | EP3031032A4 (en) |
| CN (1) | CN104603833B (en) |
| IL (1) | IL236899A0 (en) |
| WO (1) | WO2014024197A1 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107728965A (en) * | 2017-10-23 | 2018-02-23 | 上海斐讯数据通信技术有限公司 | The method and system of word in a kind of bar code printer automatic printing |
| CN111727440A (en) * | 2017-10-30 | 2020-09-29 | 脸谱公司 | System and method for determining a digital destination based on a multi-part identifier |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9355313B2 (en) * | 2014-03-11 | 2016-05-31 | Microsoft Technology Licensing, Llc | Detecting and extracting image document components to create flow document |
| WO2015152951A1 (en) | 2014-03-31 | 2015-10-08 | Hewlett-Packard Development Company, L.P. | Media navigation recommendations |
| WO2015167499A1 (en) | 2014-04-30 | 2015-11-05 | Hewlett-Packard Development Company, L.P. | Response to a machine-readable link |
| WO2016025015A1 (en) | 2014-08-11 | 2016-02-18 | Hewlett-Packard Development Company, L.P. | Media hotspot payoffs with alternatives lists |
| US10664647B2 (en) * | 2014-10-28 | 2020-05-26 | Kuang-Yao Lee | Visual web page analysis system and method |
| US10810277B1 (en) | 2017-10-30 | 2020-10-20 | Facebook, Inc. | System and method for determination of a digital destination based on a multi-part identifier |
| EP3477538A1 (en) * | 2017-10-30 | 2019-05-01 | Facebook, Inc. | System and method for determination of a digital destination based on a multi-part identifier |
| CN110163050B (en) * | 2018-07-23 | 2022-09-27 | 腾讯科技（深圳）有限公司 | Video processing method and device, terminal equipment, server and storage medium |
| US11151370B2 (en) * | 2018-11-14 | 2021-10-19 | Adobe Inc. | Text wrap detection |
| US11176311B1 (en) * | 2020-07-09 | 2021-11-16 | International Business Machines Corporation | Enhanced section detection using a combination of object detection with heuristics |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070050360A1 (en) * | 2005-08-23 | 2007-03-01 | Hull Jonathan J | Triggering applications based on a captured text in a mixed media environment |
| CN101292259A (en) * | 2005-08-23 | 2008-10-22 | 株式会社理光 | Method and system for image matching in a mixed media environment |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6138151A (en) | 1996-09-23 | 2000-10-24 | Motorola, Inc. | Network navigation method for printed articles by using embedded codes for article-associated links |
| US8176054B2 (en) * | 2007-07-12 | 2012-05-08 | Ricoh Co. Ltd | Retrieving electronic documents by converting them to synthetic text |
| US8600989B2 (en) * | 2004-10-01 | 2013-12-03 | Ricoh Co., Ltd. | Method and system for image matching in a mixed media environment |
| US8335789B2 (en) * | 2004-10-01 | 2012-12-18 | Ricoh Co., Ltd. | Method and system for document fingerprint matching in a mixed media environment |
| US8385589B2 (en) | 2008-05-15 | 2013-02-26 | Berna Erol | Web-based content detection in images, extraction and recognition |
| WO2007023993A1 (en) * | 2005-08-23 | 2007-03-01 | Ricoh Company, Ltd. | Data organization and access for mixed media document system |
| WO2007130688A2 (en) * | 2006-05-10 | 2007-11-15 | Evolution Robotics, Inc. | Mobile computing device with imaging capability |
| US20090046320A1 (en) * | 2007-08-19 | 2009-02-19 | Xsights Media Ltd. | Method and apparatus for forwarding media... |
| KR101165359B1 (en) * | 2011-02-21 | 2012-07-12 | (주)엔써즈 | Apparatus and method for analyzing relation with image and image or video |
-
2013
- 2013-08-08 WO PCT/IL2013/050679 patent/WO2014024197A1/en active Application Filing
- 2013-08-08 CN CN201380040688.XA patent/CN104603833B/en not_active Expired - Fee Related
- 2013-08-08 EP EP13827861.9A patent/EP3031032A4/en not_active Withdrawn
- 2013-08-08 US US14/419,172 patent/US9916499B2/en not_active Expired - Fee Related
-
2015
- 2015-01-25 IL IL236899A patent/IL236899A0/en unknown
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070050360A1 (en) * | 2005-08-23 | 2007-03-01 | Hull Jonathan J | Triggering applications based on a captured text in a mixed media environment |
| CN101292259A (en) * | 2005-08-23 | 2008-10-22 | 株式会社理光 | Method and system for image matching in a mixed media environment |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107728965A (en) * | 2017-10-23 | 2018-02-23 | 上海斐讯数据通信技术有限公司 | The method and system of word in a kind of bar code printer automatic printing |
| CN111727440A (en) * | 2017-10-30 | 2020-09-29 | 脸谱公司 | System and method for determining a digital destination based on a multi-part identifier |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104603833B (en) | 2018-12-14 |
| WO2014024197A1 (en) | 2014-02-13 |
| EP3031032A1 (en) | 2016-06-15 |
| EP3031032A4 (en) | 2017-03-29 |
| US9916499B2 (en) | 2018-03-13 |
| US20150242684A1 (en) | 2015-08-27 |
| IL236899A0 (en) | 2015-03-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104603833A (en) | A method and system for linking printed objects with electronic content | |
| CN109716327B (en) | Video capturing framework for visual search platform | |
| JP5972468B2 (en) | Detect labels from images | |
| US7672543B2 (en) | Triggering applications based on a captured text in a mixed media environment | |
| US7991778B2 (en) | Triggering actions with captured input in a mixed media environment | |
| US7920759B2 (en) | Triggering applications for distributed action execution and use of mixed media recognition as a control input | |
| US7587412B2 (en) | Mixed media reality brokerage network and methods of use | |
| US7551780B2 (en) | System and method for using individualized mixed document | |
| Erol et al. | HOTPAPER: multimedia interaction with paper using mobile phones | |
| US7769772B2 (en) | Mixed media reality brokerage network with layout-independent recognition | |
| US8949287B2 (en) | Embedding hot spots in imaged documents | |
| US7639387B2 (en) | Authoring tools using a mixed media environment | |
| US9171202B2 (en) | Data organization and access for mixed media document system | |
| US9405751B2 (en) | Database for mixed media document system | |
| US8838591B2 (en) | Embedding hot spots in electronic documents | |
| JP5095535B2 (en) | Image processing method, image processing system, image processing apparatus, and program | |
| JP5181887B2 (en) | System and method for collating electronic documents | |
| US7885955B2 (en) | Shared document annotation | |
| JP5181886B2 (en) | Electronic document search method, search system, and computer program | |
| EP1917636B1 (en) | Method and system for image matching in a mixed media environment | |
| JP2009043242A (en) | Method and system for generating graphical user interface | |
| US9177199B2 (en) | Semantic magazine pages | |
| JP6365024B2 (en) | Service providing apparatus, method, and program | |
| Raj et al. | Zernike-moments-based shape descriptors for pattern recognition and classification applications | |
| JP5967036B2 (en) | Image search system, information processing apparatus, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| EXSB | Decision made by sipo to initiate substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20181214 Termination date: 20190808 |