CN103377293A - Holographic touch interactive exhibition system with multisource input and intelligent information optimizing functions - Google Patents

Abstract

The invention discloses a holographic touch interactive exhibition system with multisource input and intelligent information optimizing functions. The system senses surrounding environment information by the aid of a plurality of sensors, a smoke sensor is started to monitor fire without audiences, faces in videos are recognized when the audiences are present, proper broadcasting information is recommended, and the identity sense of customers is enhanced. The exhibition system introduces an interactive mode, and the audiences can realize man-machine interaction by the aid of a holographic touch screen to acquire information that the audiences want to know. Besides, by the aid of an intelligent information mining algorithm and features extracted by a color co-occurrence matrix, finding of a facial expression recognition area of interest and feature extraction are combined into one step, the faces are clustered by the aid of an affinity propagation clustering algorithm, and the broadcasting information is changed according to emotion. By the aid of the exhibition system, the information can be effectively pushed according to ages and genders of the audiences in a complicated background environment, sufficient exhibition can be realized, the audiences can be guided to more comprehensively know the information, and the identity sense of the audiences is enhanced.

Description

The holographic touch interactive display system of multi-source input, information intelligent optimization process

Technical field

The invention belongs to the multimedia information dissemination field, particularly the holographic touch interactive display system of a kind of multi-source input, information intelligent optimization process.

Background technology

Along with the fast development of modern society, multimedia system just is being penetrated into the every field of life with extremely strong vitality, such as advertisement, game, education, archives, books, amusement, art, financial transaction etc.Advertising display is subject to the great attention of government bodies, enterprises and institutions and every profession and trade individuality as a kind of technology that conducts promotion to flexibly.It not only can spreads out merchandise information, Branding image, the aesthetic requirement that satisfies the consumer, guide consumption guiding, and it can also excite people's moral consciousness simultaneously, causes thinking, the effect that education is arranged and inspire.Present advertising display technology is mainly considered the unidirectional propelling movement of advertisement, relies on existence and repeatability to reach the effect of publicizing widely.Particularly, giant-screen LCD information display system (such as building elevator LCD ad system) is used very extensive at present.It has powerful video, animation, text ability to play, but these systems all are unidirectional, and audient colony can only passive viewing.Therefore current display systems much exists colony to promote mistake, even the stage of offensive detest, does not consider that fully the audient experiences and demand, does not more consider audient's participation.Simultaneously, to consume cost huge, cause the significant wastage of social electric power, space, human resources.

China Patent No.: CN101620715A discloses a kind of intelligent advertisement dissemination method.It scans picture or video, and picture or video is identified by picture or the video of input scene, extracts user's characteristic information; Whether the user who relatively gathers or special user's characteristic information mate with the user's characteristic information that prestores; Search the advertisement that is complementary with this user's characteristic information in the advertising resource storehouse; The output advertisement.

China Patent No.: CN102129644A discloses the intelligent advertisement system of a kind of audient's of having characteristic perception and statistical function.It is by the camera collection image, and the image information that collects sent into computing equipment, computing equipment be responsible for the image that obtains from collecting device process, computing and realize storage, demonstration, broadcast and the switching of ad content according to operation result control electric advertisement screen.

There is following weak point in above-mentioned ad system:

(1) above-mentioned patent mainly is to utilize the form of image library or image dictionary that the people's face in video or the scene picture is mated, determine audient's uniqueness and uniqueness, be not suitable for the show surroundings that complexity or audient change, in the large situation of image library data volume, computing machine is required height simultaneously, can not satisfy the requirement of system real time.

(2) fail audient's mood is taken into account, can not adjust according to audient's mood self-adaptation the broadcast of information.

(3) fail to change the state of audient's passive receive information, the client can't be by in person operation, with Media, obtains wanting the information of knowing.

(4) having in audient's the situation, do not having suitable method to promote customer share; In the situation without the audient, the raising system works efficient of failing to economize on resources.

Summary of the invention

To the objective of the invention is in order addressing the above problem, the holographic touch interactive display system of a kind of multi-source input, information intelligent optimization process to be provided.This system in the situation without the audient, starts the generation of cigarette propagated sensation sensor monitoring fire by multisensor perception ambient condition information; In the situation that has the audient to exist, the people's face in the video is identified, recommend suitable information broadcasting, increase client's acceptance.This display systems is introduced interactive mode, and the audient can realize human-computer interaction by the holographic touch screen, and the inquiry client wants the information known.Simultaneously, the feature and the intelligent information mining algorithm that extract by the color co-occurrence matrix, searching area-of-interest and the feature extraction of human face expression identification are combined into a step, utilize neighbour's propagation clustering algorithm that people's face is carried out cluster, according to mood change information broadcasting information.

Technical solution of the present invention is as follows:

The holographic interaction that a kind of multi-source information input, Intelligent Recognition are processed touches display systems, comprises that the multisensor, computing equipment and the holographic interaction that gather environmental information touch display device,

(1) open multisensor awareness apparatus monitoring surrounding enviroment, whether detection infrared sensor, sound transducer exist signal.In the situation of infrared sensor, the equal no signal of sound transducer, open the generation of cigarette sense sensor monitoring of environmental fire news feelings; Have in the situation of signal at infrared sensor or sound transducer, start camera and interactive touch display device;

It can be the movable platform that is placed on indoor or outdoors that holographic interaction touches display systems, therefore its environment of living in may be changeable, use video capture device very uneconomical round-the-clockly, therefore in the situation without the audient, display platform is closed the screen harvester, stop the broadcast of information, open environment monitoring device, prevent the generation of the dangerous situations such as fire.

(2) reading images frame or sequence of image frames from screen information.People's face detects and the location, specifically determines whether there is people's face in the input picture, if there is people's face, then determines the position of one or more people's faces and size etc.Facial feature extraction is on the face extraction of the people who the detects feature relevant with sex, age and mood.Feature commonly used comprises global feature and the local feature take the position of face and shape profile information as representative take color and Luminance Distribution information as representative.According to the facial characteristics that extracts, age and the sex of people's face in the image are made estimation.

(3) adopting intelligent information mining algorithm and neighbour's propagation clustering algorithm that human face expression is carried out INTELLIGENT IDENTIFICATION is also classified.By feature and intelligent information mining algorithm that the color co-occurrence matrix extracts, searching area-of-interest and feature extraction that human face expression is identified are combined into a step, utilize neighbour's propagation clustering algorithm that people's face is carried out cluster.

(4) the information customization is thrown in and information recommendation input.Inquirer's face whether with image library in people's face coupling, have no touch perceptual signal and human face expression whether positive? do not mate at people's face and image library, the no touch perceptual signal, and the recommendation of carrying out information in the positive situation of human face expression is play.Having in the situation that touches perception information, stop to play current information, the touch inquire request or the broadcast audient that respond fast the audient specify information.In the situation of people's face and image library coupling, do not interrupting preferential its information interested of playing under the current just prerequisite at information broadcasting; In the not positive situation of human face expression, dynamically adjust and show the information classification.

(5) if in the unclosed situation of current displaying information, continue to show current information, otherwise, return step (1).

The multisensor of described collection environmental information comprises: one or more in camera, infrared sensor, microwave remote sensor, ultrasonic sensor, laser sensor, magnetic switch sensor, sound transducer or the cigarette propagated sensation sensor, be used for obtaining the information such as environment temperature, humidity, illumination, time, and audient's fingerprint in the measurement environment, height, body weight, blood pressure, heartbeat, body temperature etc.

The holographic interaction of processing by a kind of multi-source information input of the present invention, Intelligent Recognition touches display systems, and its hardware configuration is divided into following two parts.

One, the local area control structure.A computing machine can drive one or more touch display devices at least, and each display device can be equipped with one group of sensor senses equipment.The multisensor awareness apparatus can be camera, infrared sensor, microwave remote sensor, ultrasonic sensor, laser sensor, magnetic switch sensor, sound transducer and cigarette sense sensor.

Two, the wide area syndeton.Many computing machines can the network consisting system, can transmit in wired or wireless mode, can consist of the network topology structures such as bus-type, star-like, ring-like, tree type or mixed type.In addition, with various computing machines, external unit and database link up mutually the information handling system of composition, can realize that video display file management, application software are shared, the functions such as displaying schedule, Email and facsimile service service.

In the above-mentioned steps (2), people's face detects and the location, and feature extraction mainly is divided into following steps: at first, determine whether there is people's face in the input picture, if there is people's face, then locate fast position and the scope of one or more people's faces in image.Secondly, pretreatment image makes people's face position, size and image normalization in the facial image, and overcome block, the impact of the factors such as attitude, illumination, jewelry.Again, extract on the face the feature relevant with sex and age the people who detects.Feature commonly used comprises global feature and the local feature take the position of face and shape profile information as representative take Luminance Distribution information as representative.At last, according to the facial characteristics that extracts, the sex of people's face to be identified is judged and estimates the age.

In the above-mentioned steps (3), the feature that the color co-occurrence matrix extracts comprises redness and the green component in the expression RGB color space model, the chromatic component in the hsv color spatial model and gray component.For above-mentioned arbitrary component, can comprise a plurality of characteristic quantities again, these characteristic quantities can be the contrasts of color, energy, entropy, one or more in consistance or the correlativity etc.

In the above-mentioned steps (3), the intelligent information mining algorithm forms the disaggregation of intelligent information mining algorithm by a plurality of candidate solutions, and each candidate solution all has by area-of-interest component and characteristic component and represents that its detailed process is as follows:

1) random initializtion disaggregation;

2) calculate the evaluation function value of each candidate solution;

3) with the evaluation function value greater than the candidate solution of average ratings functional value as input, call reverse cloud model, obtain the numerical characteristic of Multidimensional Cloud Model;

4) self-adaptation is adjusted numerical characteristic;

5) with revised numerical characteristic as input, call the forward cloud model, generate new candidate solution, form disaggregation of new generation;

6) if satisfy end condition, output elite's candidate solution and evaluation function value thereof; Otherwise turn to step (2).

In the above-mentioned steps (3), neighbour's propagation clustering algorithm with all sample datas as initial representative sample, and each sample data is constantly updated Attraction Degree and the degree of membership of each point as potential cluster centre by iterative process, until automatically produce several class centers.Concrete steps are as follows:

(a) initial similarity matrix;

(b) the Attraction Degree value between the calculating sample point;

(c) the degree of membership value between the calculating sample point;

(d) upgrade Attraction Degree and degree of membership value;

(e) if iterations surpasses the maximal value of setting or stops calculating, determine class center and all kinds of sample points in the several times iteration when cluster centre does not change; Otherwise return (b).

In the above-mentioned steps (4), when information customization input and information recommend to throw in, follow following several principle: one, respond at any time touch signal; Two, not interrupt under the current just prerequisite at information broadcasting, preferential answering image library matched signal is carried out the information customization according to match information and is showed; Three, carry out information according to human facial expression information and play.In the positive situation of human face expression, continue the broadcast of current information, and recommending next round to play with the recommendation of generic attribute information; In the not positive situation of human face expression, dynamically adjust and show the information classification.

The beneficial effect that the present invention reaches:

The present invention is the holographic touch interactive display system of a kind of multi-source input, information intelligent optimization process, touching display device by many sensings input equipment, computing equipment and holographic interaction forms, modular construction is convenient to the client and is carried out as required textural association and software adjustment.The characteristics of its existing billboard can reach again the effect of film advertisement, can also make the audient participate in carrying out effective interaction, are desirable integrated communication media systems.It can age, the sex according to the audient effectively push information under the complex background environment, can also adjust information according to audient's mood and touch perception information self-adaptation.It can fully show and lead the audient to understand more comprehensively information, increases audient's acceptance.

Description of drawings

Fig. 1 overall construction drawing;

Fig. 2 principle of work process flow diagram;

The process flow diagram of Fig. 3 intelligent information mining algorithm;

Fig. 4 candidate solution coding structure figure;

The express one's feelings process flow diagram of INTELLIGENT IDENTIFICATION of Fig. 5;

Fig. 6 organizes local and the wide area annexation figure of display systems more.

Embodiment

Specifically introduce a preferred embodiment of the present invention below in conjunction with accompanying drawing 1-6.

Referring to Fig. 1, the holographic interaction of processing by a kind of multi-source information input of the present invention, Intelligent Recognition touches display systems, is used for enterprises and institutions or individual customer and issues various information, and its general structure mainly is divided into three large modules:

(1.1) acquisition module of Multiple Source Sensor signal, formed by following one or more sensors, such as camera, infrared sensor, microwave remote sensor, ultrasonic sensor, laser sensor, magnetic switch sensor, sound transducer and cigarette sense sensor.Be mainly used in the situation of audient in the monitoring of environmental, and the generation of the mishap such as fire.

(1.2) computing equipment module can be computing machine or embedded computer, is used for the information that multisensor obtains is carried out intellectual analysis and processing;

(1.3) holographic touch display device, touch-screen is combined with the LCD/LED display screen by the nanometer touch membrane, adopts the tempered glass protection in the time of suitably, supports LAN (Local Area Network) or internet Long-distance Control, and accurately touch-control and Query Information are interactive with the audient.

Referring to Fig. 2, the holographic interaction of processing by a kind of multi-source information input of the present invention, Intelligent Recognition touches display systems, and its principle of work flow process comprises the steps:

(2.1) open multisensor awareness apparatus monitoring surrounding enviroment, whether detection infrared sensor, sound transducer exist signal.In the situation of infrared sensor, the equal no signal of sound transducer, open cigarette sense sensor monitoring of environmental fire news feelings; Have in the situation of signal at infrared sensor or sound transducer, start camera and interactive touch presentation device;

(2.2) reading images frame or sequence of image frames from screen information.

(2.3) people's face detects and the location, specifically determines whether there is people's face in the input picture, if there is people's face, then determines the position of one or more people's faces and size etc.Facial feature extraction is on the face extraction of the people who the detects feature relevant with sex, age and mood.Feature commonly used comprises global feature and the local feature take the position of face and shape profile information as representative take color and Luminance Distribution information as representative.

(2.4) according to the facial characteristics that extracts, age and the sex of people's face in the image are made estimation, expression and the classification of identification people face.

(2.5) inquirer's face whether with image library in people's face coupling, have no touch perceptual signal and human face expression whether positive? do not mate at people's face and image library, the no touch perceptual signal, and the recommendation of carrying out information in the positive situation of human face expression is play.Having in the situation that touches perception information, stop to play current information, the touch inquire request or the broadcast audient that respond fast the audient customize information.In the situation of people's face and image library coupling, preferentially play its information interested; In the not positive situation of human face expression, dynamically adjust and show the information classification.

(2.6) if in the unclosed situation of current displaying information, continue to show current information, otherwise, return step (2.1).

Referring to Fig. 3, the holographic interaction of processing by a kind of multi-source information input of the present invention, Intelligent Recognition touches display systems, and the workflow diagram of the intelligent information mining algorithm that it adopts comprises the steps:

(3.1) random initializtion disaggregation X (0).Be without loss of generality, consider one and contain N candidate solution, code length is candidate's disaggregation (hereinafter to be referred as disaggregation) X={x of L ₁, x ₂..., x _N∈ S ^N, wherein S is the candidate solution space encoder, S ^NBe N rank disaggregation space encoder.Disaggregation X can be with following vector and matrix representation:

$X=\left[\begin{array}{c}{x}_1\\ x_2\\ \·\\ \·\\ \·\\ x_N\end{array}\right]=\left[\begin{array}{cccccc}{x}_{11}& x_{12}& \·& \·& \·& x_{1L}\\ x_{21}& x_{22}& \·& \·& \·& x_{2L}\\ \·& \·& & \·& & \·\\ \·& \·& & \·& & \·\\ \·& \·& & \·& & \·\\ x_{N1}& x_{N2}& \·& \·& \·& x_{\mathrm{NL}}\end{array}\right]=[X_1,X_2,\·\·\·,X_L]$

X wherein _i=(x _I1, x _I2..., x _IL) ∈ S represents i candidate solution among the disaggregation X; x _IjSeparate constituent element for j that represents i candidate solution; I=1,2 ..., N, j=1,2 ..., L.

(3.2) calculate the evaluation function value f (x of each candidate solution _i(t)), disaggregation average ratings functional value

With the best evaluation function value of disaggregation be

Determine elite's candidate solution x _b(t), wherein $\stackrel{\‾}{f}\left(X\left(t\right)\right)={\mathrm{\Σ}}_{i=1}^{N}f\left(x_i\left(t\right)\right)/N,$ $\hat{f}\left(X\left(t\right)\right)=\max f\left(x_i\left(t\right)\right),$ $\left\{x_b\left(t\right)\right|f\left(x_b\left(t\right)\right)=\hat{f}\left(X\left(t\right)\right),b\∈[1,N],t\≥0\}.$

(3.3) with the evaluation function value greater than the candidate solution of average ratings functional value as input, call reverse cloud model, obtain the numerical characteristic of Multidimensional Cloud Model.Specifically can be divided into following steps again:

(3.3.1) from disaggregation X (t), select the evaluation function value greater than the candidate solution of disaggregation average ratings functional value, namely satisfy

Candidate solution x _i(t), i=1,2 ... k, wherein 1≤k≤N.

(3.3.2) with the candidate solution evaluation function f (x that selects _i(t)) be mapped to the degree of certainty of water dust $\tilde{f}\left(x_i\left(t\right)\right)\∈\left(\mathrm{0,1}\right),$ Namely

$\tilde{f}\left(x_i\left(t\right)\right)=1-\frac{f\left(x_i\left(t\right)\right)+\hat{f}\left(X\left(t\right)\right)}{\stackrel{\‾}{f}\left(X\left(t\right)\right)}-\mathrm{\ϵ}$

Wherein ε is minimum constant, to guarantee

Preventing from occuring when calling backward cloud generator logarithm overflows.

(3.3.3) will

I=1,2 ... k calls the multidimensional backward cloud generator as input, calculates the cloud numerical characteristic [Ex (t) of disaggregation; En (t); He (t)].Specifically,

$\left[\begin{array}{c}\mathrm{Ex}\left(t\right)\\ \mathrm{En}\left(t\right)\\ \mathrm{He}\left(t\right)\end{array}\right]=\left[\begin{array}{cccc}E{x}_1\left(t\right)& E{x}_2\left(t\right)& \·\·\·& E{x}_L\left(t\right)\\ E{n}_1\left(t\right)& E{n}_2\left(t\right)& \·\·\·& E{n}_L\left(t\right)\\ H{e}_1\left(t\right)& H{e}_2\left(t\right)& \·\·\·& H{e}_L\left(t\right)\end{array}\right].$

Its computation process is as follows:

$E{x}_j\left(t\right)=\frac{1}{k}\underset{i=1}{\overset{k}{\mathrm{\Σ}}}{x}_{\mathrm{ij}}\left(t\right)$

$E{n}_j\left(t\right)=\sqrt{\frac{1}{k}\underset{i=1}{\overset{k}{\mathrm{\Σ}}}{(x_{\mathrm{ij}}\left(t\right)-E{x}_j\left(t\right))}^2}$

To whenever a pair of

Calculate

${\mathrm{En}}_{\mathrm{ij}}^{\′}\left(t\right)=\sqrt{\frac{-{(x_{\mathrm{ij}}\left(t\right)-E{x}_j\left(t\right))}^2}{2\ln \tilde{f}\left(x_i\left(t\right)\right)}},$

$H{e}_j\left(t\right)=\sqrt{\frac{1}{k}\underset{i=1}{\overset{k}{\mathrm{\Σ}}}{({\mathrm{En}}_{\mathrm{ij}}^{\′}\left(t\right)-{\stackrel{\‾}{E}n}_j^{\′}\left(t\right))}^2}\·$

Wherein $\stackrel{\‾}{E}{n}_j^{\′}\left(t\right)=\frac{1}{k}\underset{i=1}{\overset{k}{\mathrm{\Σ}}}{\stackrel{\‾}{E}n}_{\mathrm{ij}}^{\′}\left(t\right).$

(3.4) with numerical characteristic as input, call the forward cloud model, generate interim disaggregation X _Temp1(t+1) also determine its elite's candidate solution x _b(t+1);

For a known multidimensional numerical characteristic [Ex (t); En (t); He (t)], t 〉=0 according to the forward cloud model, produces the new candidate solution x with this problem uncertain knowledge to be optimized _i(t+1), i=1,2 ... N.Its process is as follows:

(3.4.1) generating an expectation value is En (t), and standard deviation is the multidimensional normal random number of He (t)

En′ _i(t)=NormRand(En(t),He(t)).

(3.4.2) generating an expectation value is Ex (t), and standard deviation is En ' _i(t) multidimensional normal random number

y _i(t)=NormRand(Ex(t),En′ _i(t)).

(3.4.3) produce formation disaggregation X _Temp1(t+1) a candidate solution x _iAnd x (t+1), _i(t+1)=y _i(t);

(3.4.4) calculate x _i(t+1) evaluation function value f (x _i(t+1));

(3.4.5) repeat (3.4.1) to (3.4.4), until generate till N the candidate solution.

(3.5) if continuous span occurs for the elite, namely satisfy Produce at random new candidate's disaggregation

(3.6) from disaggregation X (t) and

Middle selection evaluation function value is called reverse cloud model greater than the candidate solution of disaggregation average ratings functional value, sets up new disaggregation characteristic parameter

(3.7) with revised numerical characteristic

As input, call the forward cloud model, generate new interim disaggregation X _Temp2(t+1);

(3.8) at interim disaggregation X _Temp1(t+1) and X _Temp2(t+1) select the evaluation function value to be not less than in

Candidate solution, form disaggregation X of new generation (t+1);

(3.9) if satisfy end condition, output elite's candidate solution and the conduct of evaluation function value thereof; Otherwise turn to step (3.2).

Referring to Fig. 4, the holographic interaction of processing by a kind of multi-source information input of the present invention, Intelligent Recognition touches display systems, adopt computing intelligence that selection and the feature extraction of area-of-interest in video or the image are organically combined, its candidate solution coding structure comprises following part:

(4.1) area-of-interest C ₁, C ₂..., C _i... C _m, C wherein _iRepresent the selection of i area-of-interest, m represents the number of area-of-interest.

Specifically for area-of-interest C _i, its coding can be subdivided into α again _i, x _Ci, y _Ci, h _Ci, w _Ci, α wherein _iThe C that expression is corresponding _iWhether be activated x _Ci, y _CiRepresent respectively area-of-interest C _iThe horizontal ordinate of central point and ordinate, h _Ci, w _CiRepresent respectively area-of-interest C _iHeight and width.

(4.2) proper vector of extraction area-of-interest

Wherein

Redness in the expression RGB color space model and green; Colourity in the expression hsv color spatial model;

The gray component of expression area-of-interest.

For above-mentioned arbitrary component, can comprise a plurality of characteristic quantities again.First with

Be example, These characteristic quantities can be the contrasts of color, energy, entropy, consistance, correlativity etc.

Referring to Fig. 5, the holographic interaction of processing by a kind of multi-source information input of the present invention, Intelligent Recognition touches display systems, and the process flow diagram of its expression INTELLIGENT IDENTIFICATION mainly is divided into such as lower module:

(5.1) intelligent information mining algorithm module.The disaggregation of the intelligent information mining algorithm that is made of a plurality of candidate solutions, each candidate solution all have by area-of-interest component and characteristic component and represent, specifically referring to Fig. 4.The disaggregation renewal process of intelligent information mining algorithm is referring to Fig. 3.

(5.2) color co-occurrence matrix parameter is selected module.Color co-occurrence matrix parameter comprises pel spacing d and direction θ, if consider that one 8 is communicated with neighborhood, gets d _{γ i}=1, θ _{γ i}=90 °.The intelligent information mining algorithm carries out feature construction by the parameter of adjusting the color co-occurrence matrix, and concrete steps are

(5.2.1) color levels with image is defined as the L level, and color co-occurrence matrix M (i, j) is one L * L matrix, the capable j column element of i m wherein _IjThe expression gray scale be i and gray scale be two of j at a distance of the number of times of pixel to occurring for (Δ x, Δ y).To the arbitrary regional R in the image, definition H is for wherein having the right set of pixel of particular space contact, and the color co-occurrence matrix of the area-of-interest that activates in each candidate solution so can be expressed as

M(i,j)=#{[(x ₁,y ₁),(x ₂,y ₂)∈H|f(x ₁,y ₁)=i&f(x ₂,y ₂)=j]}

$\hat{M}(i,j)=\frac{M(i,j)}{\#\left(H\right)}$

Wherein, i ∈ [0, L-1], j ∈ [0, L-1], x ₂=x ₁+ dcos θ, y ₂=y ₁+ dsin θ, among # (H) the expression set H to the number of the contributive element of M (i, j).

(5.2.2) usually too much can cause dimension disaster for regional R pixel, usually choose a plurality of characteristic quantities and represent.With the green in the RGB color space model

Be example,

(a) contrast: ${\mathrm{\γ}}_{G1}=\underset{i=0}{\overset{L-1}{\mathrm{\Σ}}}\underset{j=0}{\overset{L-1}{\mathrm{\Σ}}}{(i-j)}^2\hat{M}(i,j)$

(b) energy: ${\mathrm{\γ}}_{G2}=\underset{i=0}{\overset{L-1}{\mathrm{\Σ}}}\underset{j=0}{\overset{L-1}{\mathrm{\Σ}}}{\left[\hat{M}(i,j)\right]}^2$

(c) entropy: ${\mathrm{\γ}}_{G3}=-\underset{i=0}{\overset{L-1}{\mathrm{\Σ}}}\underset{j=0}{\overset{L-1}{\mathrm{\Σ}}}\hat{M}(i,j)\log \hat{M}(i,j),$ Wherein work as $\hat{M}(i,j)=0$ The time, $\log \hat{M}(i,j)=0$

(d) consistance: ${\mathrm{\γ}}_{G4}=-\underset{i=0}{\overset{L-1}{\mathrm{\Σ}}}\underset{j=0}{\overset{L-1}{\mathrm{\Σ}}}\hat{M}(i,j)/[1+{(i-j)}^2]$

(e) correlativity: ${\mathrm{\γ}}_{G5}=-\underset{i=0}{\overset{L-1}{\mathrm{\Σ}}}\underset{j=0}{\overset{L-1}{\mathrm{\Σ}}}\mathrm{ij}\hat{M}(i,j)-{\mathrm{\μ}}_1{\mathrm{\μ}}_2/{\mathrm{\σ}}_1^2{\mathrm{\σ}}_2^2,$ Wherein ${\mathrm{\μ}}_1=\underset{i=0}{\overset{L-1}{\mathrm{\Σ}}}\underset{j=0}{\overset{L-1}{\mathrm{\Σ}}}\hat{M}(i,j),$ ${\mathrm{\μ}}_2=\underset{j=0}{\overset{L-1}{\mathrm{\Σ}}}j\underset{i=0}{\overset{L-1}{\mathrm{\Σ}}}\hat{M}(i,j),$ ${\mathrm{\σ}}_1^2=\underset{i=0}{\overset{L-1}{\mathrm{\Σ}}}{(i-{\mathrm{\μ}}_1)}^2\underset{j=0}{\overset{L-1}{\mathrm{\Σ}}}\hat{M}(i,j),$ ${\mathrm{\σ}}_2^2=\underset{j=0}{\overset{L-1}{\mathrm{\Σ}}}{(j-{\mathrm{\μ}}_1)}^2\underset{i=0}{\overset{L-1}{\mathrm{\Σ}}}\hat{M}(i,j).$

(5.3) neighbour's propagation clustering module.As initial representative sample, and each sample data constantly updates Attraction Degree and the degree of membership of each point as potential cluster centre (exemplar) by iterative process to this module with all sample datas, until automatically produce several class centers.Concrete steps are as follows

(5.3.1) initial similarity matrix S, i capable j column element S (i, j) expression sample point i wherein and the similarity between the sample point j are to the P assignment.

(5.3.2) the Attraction Degree value between the calculating sample point

$R(i,k)\←S(i,k)-\underset{j\≠k}{\max }[S(i,j)+A(i,j)],$

Wherein A (i, j) expression sample point j is for the degree of membership value of sample point i.

(5.3.3) the degree of membership value between the calculating sample point

$A(i,k)\←\min \{0,R(k,k)+\underset{j\≠i,k}{\max }[0,R(j,k)\left]\right\},$

$A(k,k)\←\underset{j\≠k}{\max }[0,R(j,k)]$

(5.3.4) upgrade Attraction Degree and degree of membership value

$R_{i+1}(i,k)=\mathrm{\ρ}\·R_i(i,k)+(1-\mathrm{\ρ})\·R_{i+1}^{\mathrm{old}}(i,k),\mathrm{\ρ}\∈\left[\mathrm{0.5,1}\right)$

$A_{i+1}(i,k)=\mathrm{\ρ}\·A_i(i,k)+(1-\mathrm{\ρ})\·A_{i+1}^{\mathrm{old}}(i,k),\mathrm{\ρ}\∈\left[\mathrm{0.5,1}\right)$

Wherein ρ is convergence coefficient, is mainly used in regulating the stability of convergence of algorithm speed and iterative process.

$A_{i+1}(k,k)=P\left(k\right)-\max [A_{i+1}(k,j)+S_{i+1}(k,j)],j\∈\{\mathrm{1,2},\·\·\·,N\},j\≠k$

If (5.3.5) iterations surpasses the maximal value of setting or stops calculating, determine class center and all kinds of sample points when cluster centre does not change in the several times iteration; Otherwise return (5.3.2).

Referring to Fig. 6, the holographic interaction of processing by a kind of multi-source information input of the present invention, Intelligent Recognition touches display systems, local and the wide area annexation figure of its many group display systems, and its primary structure is divided into following two parts.

(6.1) local area control structure.A computing machine can drive one or more touch display devices, and each display device can be equipped with one group of sensor senses equipment.The multisensor awareness apparatus can be camera, infrared sensor, microwave remote sensor, ultrasonic sensor, laser sensor, magnetic switch sensor, sound transducer and cigarette sense sensor.

(6.2) wide area syndeton.Many computing machines can the network consisting system, can transmit in wired or wireless mode, can consist of the network topology structures such as bus-type, star-like, ring-like, tree type, mixed type.In addition, with various computing machines, external unit and database link up mutually the information handling system of composition, can realize that video display file management, application software are shared, the functions such as displaying schedule, Email and facsimile service service.

The present invention is that the holographic interaction that a kind of multi-source information input, Intelligent Recognition are processed touches display systems, touching display device by many sensings input equipment, computing equipment and holographic interaction forms, modular construction is convenient to the client and is carried out as required textural association and software adjustment.The characteristics of its existing billboard can reach again the effect of film advertisement, can also make the audient participate in carrying out effective interaction, are desirable integrated communication media systems.It can the feature according to the audient effectively push information under the complex background environment, can also adjust information according to audient's mood and touch perception information self-adaptation.It can fully show and lead the audient to understand more comprehensively information, increases audient's acceptance.

Above-described embodiment only is used for explanation the present invention, but not limitation of the invention, conversion and modification to above-described embodiment in connotation scope of the present invention all should drop in the scope of claim of the present invention.

Claims (8)

1. the holographic touch interactive display system of multi-source input, information intelligent optimization process is characterized in that, comprises the multisensor, computing equipment and the holographic interaction touch display device that gather environmental information,

(1) open multisensor awareness apparatus monitoring surrounding enviroment, whether detection infrared sensor, sound transducer exist signal; In the situation of infrared sensor, the equal no signal of sound transducer, open the generation of cigarette propagated sensation sensor monitoring of environmental fire news feelings; Have in the situation of signal at infrared sensor or sound transducer, start camera and interactive touch display device;

(2) reading images frame or sequence of image frames from screen information: people's face detects and the location, specifically determines whether there is people's face in the input picture, if there is people's face, then determines position and the size of one or more people's faces; Facial feature extraction is extracted the feature relevant with sex and age on the face the people who detects; According to the facial characteristics that extracts, age and the sex of people's face in the image are made estimation;

(3) adopting intelligent information mining algorithm and neighbour's propagation clustering algorithm that human face expression is carried out INTELLIGENT IDENTIFICATION is also classified; By feature and intelligent information mining algorithm that the color co-occurrence matrix extracts, searching area-of-interest and feature extraction that human face expression is identified are combined into a step, utilize neighbour's propagation clustering algorithm that people's face is carried out cluster;

(4) information customization is thrown in and information recommends to throw in: inquirer's face whether with image library in people's face mate, have no touch perceptual signal and human face expression whether positive; Do not mate at people's face and image library, the no touch perceptual signal, and the recommendation of carrying out information in the positive situation of human face expression is play; Having in the situation that touches perception information, stop to play current information, the touch inquire request or the broadcast audient that respond fast the audient specify information; In the situation of people's face and image library coupling, do not interrupting preferential its information interested of playing under the current just prerequisite at information broadcasting; In the not positive situation of human face expression, dynamically adjust and show the information classification;

(5) if in the unclosed situation of current displaying information, continue to show current information, otherwise, return step (1).

2. the holographic touch interactive display system of multi-source as claimed in claim 1 input, information intelligent optimization process, it is characterized in that the multisensor of described collection environmental information comprises: multiple in camera, infrared sensor, microwave remote sensor, ultrasonic sensor, laser sensor, magnetic switch sensor, sound transducer or the cigarette propagated sensation sensor.

3. the holographic touch interactive display system of multi-source as claimed in claim 1 input, information intelligent optimization process, it is characterized in that, display systems comprises following two parts hardware configuration: one, the local area control structure: a computing machine drives one or more touch display devices, and each display device is equipped with one group of sensor at least; Two, the wide area syndeton: many computing machine network consisting systems, transmit in wired or wireless mode, consist of bus-type, star-like, ring-like, tree type or heterogeneous network topological structure.

4. the holographic touch interactive display system of multi-source as claimed in claim 1 input, information intelligent optimization process, it is characterized in that, in the described step (2), people's face detects and the location, feature extraction may further comprise the steps: at first, determine whether there is people's face in the input picture, if there is people's face, then locate fast position and the scope of one or more people's faces in image; Secondly, pretreatment image makes people's face position, size and image normalization in the facial image, and overcome block, the impact of attitude, illumination, jewelry factor; Again, extract on the face the feature relevant with sex and age the people who detects; Feature commonly used comprises global feature and the local feature take the position of face and shape profile information as representative take Luminance Distribution information as representative; At last, according to the facial characteristics that extracts, the sex of people's face to be identified is judged and estimates the age.

5. the holographic touch interactive display system of multi-source as claimed in claim 1 input, information intelligent optimization process, it is characterized in that, in the described step (3), the feature that the color co-occurrence matrix extracts comprises redness and the green component in the expression RGB color space model, the chromatic component in the hsv color spatial model and gray component; For above-mentioned arbitrary component, comprise a plurality of characteristic quantities, these characteristic quantities are one or more in contrast, energy, entropy, consistance or the correlativity of color.

6. the holographic touch interactive display system of multi-source as claimed in claim 1 input, information intelligent optimization process, it is characterized in that, in the described step (3), the intelligent information mining algorithm, the disaggregation that is formed the intelligent information mining algorithm by a plurality of candidate solutions, each candidate solution all has by area-of-interest component and characteristic component and represents that its detailed process is as follows:

(61) random initializtion disaggregation;

(62) calculate the evaluation function value of each candidate solution;

(63) with the evaluation function value greater than the candidate solution of average ratings functional value as input, call reverse cloud model, obtain the numerical characteristic of Multidimensional Cloud Model;

(64) self-adaptation is adjusted numerical characteristic;

(65) with revised numerical characteristic as input, call the forward cloud model, generate new candidate solution, form disaggregation of new generation;

(66) if satisfy end condition, output elite's candidate solution and evaluation function value thereof; Otherwise turn to step (62).

7. the holographic touch interactive display system of multi-source as claimed in claim 1 input, information intelligent optimization process, it is characterized in that, in the described step (3), neighbour's propagation clustering algorithm with all sample datas as initial representative sample, and each sample data is as potential cluster centre, constantly update Attraction Degree and the degree of membership of each point by iterative process, until automatically produce several class centers; Concrete steps are as follows:

(71) initial similarity matrix;

(72) the Attraction Degree value between the calculating sample point;

(73) the degree of membership value between the calculating sample point;

(74) upgrade Attraction Degree and degree of membership value;

(75) if iterations surpasses the maximal value of setting or stops calculating, determine class center and all kinds of sample points in the several times iteration when cluster centre does not change; Otherwise return (72).

8. the holographic touch interactive display system of multi-source input as claimed in claim 1, information intelligent optimization process is characterized in that, in the described step (4), when information customization input and information recommend to throw in: one, respond at any time touch signal; Two, not interrupt under the current just prerequisite at information broadcasting, preferential answering image library matched signal is carried out the information customization according to match information and is showed; Three, carry out information according to human facial expression information and play; In the positive situation of human face expression, continue the broadcast of current information, and recommending next round to play with the recommendation of generic attribute information; In the not positive situation of human face expression, dynamically adjust and show the information classification.

Images