Classification with Artificial Neural Networks and Support Vector Machines application to o

ß2007Millpress,Rotterdam,ISBN978-90-5966-053-3

ClassificationwithArtificialNeuralNetworksandSupportVectorMachines:applicationtooilfluorescencespectra

K.M.Almhdi1,3,P.Valigi1,V.Gulbinas2,R.Westphal3&R.Reuter3Universita’diPerugina,DipartimentodiIngegneriaElettronicaedell’Informazione,06125Perugia,Italy2InstituteofPhysics,2053Vilnius,Lithuania3¨tOldenburg,Institutfu¨rPhysik,26111Oldenburg,GermanyCarlvonOssietzkyUniversita

1Keywords:ArtificialNeuralNetworks,multilayerperceptron,SupportVectorMachines,

andOilfluorescence

ABSTRACT:Thispaperreportsonoilclassificationwithfluorescencespectroscopy.Theinvestigationsarepartofthedevelopmentofalaser-basedremotesensor(laserfluorosensor)tobeusedforthedetectionandclassificationofoilspillsonwatersurfaces.ThepolychromatorofthefluorosensorhassixchannelsformeasuringsignalsthatrepresentthefluorescencespectralsignatureofthedetectedoilintheUV/VISwavelengthrangefollowingexcitationat355nm.Theinvestigationoftheoilclassifica-tionisbasedontheshapeofthesignatureoftheoildetectedbythesechannels.Theinvestigationusesthreemethodstoexaminecrudeoils,heavyrefinedoils,andsludgeoils:thechannelsrelationshipsmethod(CRM);artificialneuralnetworks(ANN);andsupportvectormachines(SVM).Thiswasdonebasedonalaboratorydatabaseofoilfluorescencespectra.

Thedatabaseandtheinputfluorescencesignatureoftheoilsplayaveryimportantroleintheefficiencyoftheclassificationmethod.Iftheinputfluorescenceoftheoildoesnotfitintooneoftheclassesalreadyincludedinthedatabaseorifitisacompletelynewandpreviouslynotconsideredsignature,thentheclassificationmethodmustalwaysberedone.Generally,allthreemethodsyieldpromisingresultsandcanbeusedforthedetectionandclassificationofoilspillsonwatersurfaces.Thechannelsrelationshipmethodprovidesameaningfulclassificationoftheavailablespectra,accordingtoaroughoiltypeestimation.MorespecificsubstanceinformationcouldbeachievedwithANNsandSVMs.BothSVMsandANNsprovetobeefficient,fastandreliableandhavereal-timecapabilities.TheSVMmethodisfasterandmorestablethanANN.Therefore,itisconsideredtobethemostconvenientmethodforclassifyingspectralinformation.

1INTRODUCTION

Artificialneuralnetworks(ANN)havebeeninvolvedinmanyapplicationstosolverealworldproblems.IncommercialpurposesANNscanbeappliedtopredicttheprofit,marketmovements,andpricelevelbasedonthemarket’shistoricaldataset.Inmedicalapplications,doctorscanevaluatethecaseofmanypatientsdependingonthehistoricaldatasetofotherpatientswhohadthesamecase.Inindustry,engineerscanapplyANNs

413

tosolvemanyengineeringproblemssuchasclassifications,prediction,patternrecogni-tion,andnon-linearproblemswheretheissuesareverydifficultormightbeimpossibletosolvethroughnormalmathematicalprocesses.ANNshavebeenappliedtopredictslantpathrainattenuation(HongweiYang,ChenHe,HongwenZhu&WentaoSong2000),topredictrainattenuationonanEarth-spacepath,topredictwaterqualityindex(WQI),andtosignalpredictionsinanuclearpowerplant(KimWJ,SHChang&BHLee1993).Theyhavealsobeenusedinfacerecognition(MingZhang&JFulcher1996).Inmedicalapplications,ANNshavebeenutilizedindetectingbraindisease(JervisBW,MRSaatchi,ALacey,TRoberts,EMAllen,NRHudson,SOke&MGrimsley1994)andDNAploidy,aswellascellcycledistributionofbreastcanceraspiratecellsthataremeasuredbyimagecytometryandanalysedbyANNsfortheirprognosticsignificance(NaguibRNG,HAMSakim,MSLakshmi,VWadehra,TWJLennard,JBhatavdekar&GVSherbet1999).

Supportvectormachines(SVM)aremodernandeffectivetoolsthathavealreadybeenexaminedtosolvedifficultiessuchasclassificationproblemsandpatternrecognition.SVMscanbeusedtosolvemorecomplexproblemswhencomparedtoANNs.InSVMsthereisnoneedtoselectfeaturesfromseveralapplications,andSVMshavedemonstratedthattheyaremoreaccurateandstablethanANNs,whichwillbeprovenforfluorescencespectraclassificationlaterinthispaper.SVMshavebeenappliedtomedicalbinaryclassificationproblems(YuchunTang,BoJin,YiSun&Yan-QingZhang2004),torecognizeradaremittersignals(GexiangZhang,WeidongJin&LaizhaoHu2004),todetectcomplicatedattacks(MingTian,Song-CanChen,YiZhuang&JiaLiu2004),andtovisualspeechrecognition(GordanM,CKotropoulos&IPitas2002),andmanyotherapplications.

Thispaperreportsonoilclassificationwithfluorescencespectroscopy.Theobjectiveistoclassifytheoilfluorescencespectrabasedonalaboratorydatasetoffluorescencespectraofseveraloilclasses(sludge,crudeandheavyoil).Theclassificationwascarriedoutusingthefollowingthreemethods:channelrelationshipmethod(CRM),artificialneuralnetworks(ANNs),andsupportvectormachines(SVMs).2METHODS

WithintheEU-fundedprojectFLUOSENSEalaserfluorosensorhasbeendevelopedforthedetectionofoilfilmsonwatersurfacesviafluorescenceexcitationoverdistancesof50to100m(KarpiczR,ADementjev,ZKuprionis,SPakalnis,RWestphal,RReuter&VGulbinas2005).TheinstrumentconsistsofaUV-emitting(355nmwavelength)pulselaserfortargetillumination,atelescopeforefficientcollectionoflightfromtheilluminatedarea,andapolychromaticgratingspectrometerequippedwithphoto-multipliersforasensitivedetectionofspectralinformationinsixUV/VISchannelsat366–395nm(1),386–425nm(2),442–486nm(3),492–552nm(4),574–4nm(5)and630–700nm(6)(Figure1).

Forexaminingtheoiltypeclassificationcapabilityofthespectrograph,adatasetof111spectraofsludge,crudeandheavyrefinedoilsmeasuredwithaPerkinElmerLS50laboratoryspectrofluorometer.Themeasuringprocedureusedforthisdatasetisthesameasdocumentedinanearlierdatacatalogue(CatalogueofOpticalSpectraof

414K.M.Almhdi,P.Valigi,V.Gulbinas,R.Westphal&R.Reuter

Figure1.Opticaldesignofthepolychromator.

Oils2005)butincludesalsothe355nmexcitationwavelengthwhichwasnotconsideredin(CatalogueofOpticalSpectraofOils2005).Theabsolutefluorescenceefficiencyoftheoilsisofminarrelevancesincethesignalintensitiesmeasuredwiththelaserfluorosensorwoulddependonotherfactorsaswell,e.g.onthetargetdistanceandtheoilfilmthickness.Therefore,spectralintensitiesarediscardedbynormalisingthespectrainthedatabasetounityoftheirintegralovertheentireemissionspectrum,andthesenormalisedspectralsignaturesareusedforclassification.Inthenextstepthenormalisedspectraareintegratedinthelimitsofthedetectionchannels2to6,tosimulatedatawhicharemeasuredbythelaserfluorosensorinthepresenceoftheseoils(Figure2).

Figure2.Examplesofoilspectraspectrallygroupedaccordingtothedetectionchannelwavebandsofthelaserfluorosensor.

ClassificationwithArtificialNeuralNetworksandSupportVectorMachines415

Channel1isnotfurtherconsideredsinceinpracticalfluorosensorapplicationsoverwaterthissignalincludesthewaterRamanscatteringwhichissefulforoilfilmthicknessmeasurements(HogeFE&RNSwift1980,HengstermannT&RReuter1990)andduetothisunderlyingsignallessspecificforoiltypeclassification.

Theclassificationiscarriedoutusingthreemethods:artificialneuralnetworks(ANNs),supportvectormachines(SVMs)andchannelrelationshipmethod(CRM).Thenormalizedfluorescencesignaturesdetectedbythechannelsofthepolychromatorareconsideredtobetheinputstotheclassificationmethodtodeterminetheiroutput(oilclass).2.1ChannelRelationshipMethod(CRM)

Thechannelrelationshipmethod(CRM)issimplemethod;itisasimpleconnectionamongtherelativeintensityofthenormalizedoilfluorescencedetectedbyeachchannelofthepolychromator.Somefluorescencesignalshaveastrongappearancewithinacertainwavelengthandaweakappearancewithintheotherrangesofthewavelength.Thefluorescencesignals,whichhaveshapessimilartoeachother,willformtheirowngroupinadiagram.Wedoexpectthatsuchcrudeoilwherethefluorescencesignaturesareclosetoeachotherwilldistinguishthemselvesamongtheotheroilclasses.Thesamerolewillbeappliedtotherestoftheoilclasseswiththeexceptionofsomeoilsfromoneormoreclassesthathaveclosesignaturestothosefromotheroilclasses,whichleadstodifficultiesinevaluatingtheclassofsuchoils.2.2ArtificialNeuralNetworks(ANN)

Artificialneuralnetworksarepowerfultoolsthatcanlearntosolveproblemsinawaysimilartohowthehumanbrainworks.ANNsgatherknowledgebydetectingthepatternsandrelationshipsindataandlearn(or:aretrained)throughexperience,notfromprogramming(Agatonovic-KustrinS&RBeresford2000).Figure3showsthestructureoftheANN.Itisacombinationofmanysingleneurons.TheANNmightconsistofseveralthousandartificialneurons,andtheoutputofoneneuronbecomesaninputtoanotherneuron.Figure4showstheneuronmodeloftheartificialneuralnetworkwheretheoutputofsuchaneuronisgivenby:

Z¼f

4X0

!

viXi(1)

Figure3.StructureofanArtificialNeuralNetwork.

416K.M.Almhdi,P.Valigi,V.Gulbinas,R.Westphal&R.Reuter

Figure4.NeuronModelofanArtificialNeuralNetwork.

ManytransferfunctionscoulddefinefðxÞ(Figure5).ThetransferfunctionusedinthisworkistheTanhfunctionwhichisamongthemostpopularfunctionstotheneuralnetworkdesignduetoitsmathematicalpropertiessuchasmonotonicity,continuity,anddifferentiability,whichareimportanttothetrainingprocess(KevinL.PriddyandPaulE.Keller2005).

ThereareseveraltypesofANNsaccordingtotheirstructureandlearningalgorithms.AccordingtotheirstructureANNscanbeclassifiedasfeedforwardnetworksandrecurrentnetworks(DucTruongPham&LiuXing1999).Inafeedforwardnetwork,theneuronsaregenerallygroupedintolayers.Signalsflowfromtheinputlayerthroughtheoutputlayerviaunidirectionalconnections,theneuronsbeingconnectedfromonelayertothenext,butnotwithinthesamelayer(DucTruongPham&LiuXing1999).Inrecurrentnetworks,theoutputofsomeneuronsisfedbacktothesameneuronsortoneuronsinaprecedinglayer(DucTruongPham&LiuXing1999).AccordingtoANNlearningprocess,theANNcanbeclassifiedtosupervisedlearning,unsupervisedlearning,andreinforcement.InthesupervisedmodeltheANNrequirestheoutputinordertoadjustitsweight.Intheunsupervisedmodel,theANNdoesnotrequiretheoutput,theANNadaptspurelyinresponsetoitsinput.Thesenetworkslearnandbuildtheirstructurebasedontheinput.Thereinforcementlearningalgorithmemploysacritictoevaluatethegoodnessoftheneuralnetworkoutputcorrespondingtoagiveninput(DucTruongPham&LiuXing1999).

InthispapertheANNshallestimatetheunknownoilclassinthelaboratorydatasetoftheseveraloilclasses.Multilayerperceptron(MLP)iswell-knowntypeofANN,whichisusuallyusedinclassificationproblems.InMLPtheneuronsaregroupedinmanylayers(Figure6).Inthiswork,MLPwithsupervisedlearninghasbeenused.Inthisapproach,duringthetrainingprocessofthenetwork,thenetworkcomparesitsactualresultsyðtÞwiththedesiredoutputdðtÞandthencomputestheerror(Eq.2).

ClassificationwithArtificialNeuralNetworksandSupportVectorMachines417

Figure5.Transferfunctionstoneuralnetworkdesign.

Figure6.MultilayerPerceptron(MLP).

418K.M.Almhdi,P.Valigi,V.Gulbinas,R.Westphal&R.Reuter

Figure7.MultilayerPerceptronusedtoclassifyoilfluorescence.

Throughthebackpropagationalgorithmtheerrorwillbepresentedmanytimestotheinputoftheforwardactivationplace,andtheprocesswillcontinueuntiltheactualoutputsgetclosertothedesiredoutput.TheMLPthatisusedinthisworkcontainsthreelayers;inputlayer,onehiddenlayerandoutputlayer(Figure7).Eq.2representstheerrorcalculatedfortheinputvectorthatispresentedtothefeedforwardnetworkandEq.3representstheerrorcalculatedforallinputvectorsthatarepresentedtothefeedforwardnetwork.

1X

ðdiðtÞÀyiðtÞÞ2

2i

vX1X

etðtÞ¼ðdiðtÞÀyiðtÞÞ2

2v¼1ieðtÞ¼

(2)(3)

Figure7showstheMLPwiththebackpropagationalgorithmthatisusedinthiswork.Thecomponentsofthisnetworkaredescribedindetailin(NeuroDimension,Inc.).Thenetworkconsistsofaninputlayer(inputAxon),anon-linearhiddenlayer(hiddenAxon),andanoutputlayer(outputAxon).Hiddenlayerandtheoutputlayerapplythetanhtransferfunction.

TheMLPnetworkhasbeentrainedbasedon111oilfluorescencesignaturesofcrudeoil,heavyoil,andsludgeoil.Thecrossvalidationofthenetworkusesadatasetofnineoilfluorescencespectra,withtheaimtoavoidover-trainingthenetwork.Thetrainednetworkthenhasbeentestedbasedonthreeoilfluorescencesignatures.Thenthreeunknownoilfluorescencespectrafromthesamethreeoilclassesusedinthetraininghavebeenfedintothetrainednetworkforclassification.TheMLPwasretrainedinordertoestimatethequalityoftheresultandtoinvestigatethestabilityofthetrainedMLP.2.3SupportVectorMachines(SVM)

SVMsarepowerfultoolsforclassificationthatcanbeconsideredasanalternativetothemulitlayerperceptron.SVMswerefirstintroducedin1992(B.E.Boseretal.,A1992).GoodexplanationforSVMstheoryandtheirapplicationsisfoundinreference(LipoWang2005).Thebasicideaistofindthelinearclassifiercalledthehyperplane.Figure8showsmanylinearclassifiersseparatetwoclasses(redandgreen).Thereisanidealseparatingclassifier(blackline)calledhyperplane(maximummarginlinearclassifier)

ClassificationwithArtificialNeuralNetworksandSupportVectorMachines419

Figure8.Thebestlinearclassifier(hyperplane).

whichcanincreasethespacebetweenitandthenearestdatasetpointsofdifferentclassesasmuchaspossible.ForthisSVMsareregardedtobemarginclassifiers.Thiscaseisasimplecasewheretheclassescanbeseparatedeasily,thiskindofsupportvectormachinesiscalledlinearsupportvectormachine(LSVM).

Forseparableclasses(Figure9),aSVMclassifiercomputesadecisionfunctionhavingamaximalmarginMwithrespecttothetwoclasses(blueandredclasses).There

Figure9.Classificationoflinearlyseparabledataset.

420K.M.Almhdi,P.Valigi,V.Gulbinas,R.Westphal&R.Reuter

aretwoplanestouchingtheboundaryofdataset,wxþb¼þ1andwxþb¼À1.wisavectorperpendicularontheplanewxþb¼þ1.Weconsiderx2tobeanypointontheplanewxþb¼þ1andx1tobetheclosestpointtox2ontheplanewxþb¼À1.Thelinefromx2tox1isperpendiculartheplanes.ThemaximummarginofthebestptoffiffiffiffiffiffiffiffiffifficlassifiercanexpressedasM¼2=wÁw.

Weconsideradataset:ðx1;y1Þ;............;ðxn;ynÞ;8i2ð1;......:;nÞ.Thedecisionboundariescanbefoundbysolvingthefollowingconstrainedoptimisingproblem:1

Minimisekwk2subjectto:

2

yiðwÁxiþbÞÀ1!0;

8i

(4)

TheLagrangeFunctionFormulationofthisoptimisingproblemisgivenby:

nX12

Lðw;b;aÞ¼kwkÀaiðyiðwÁxiþbÞÀ1Þ;

2i

ai!08i(5)

bysettingthederivativeoftheLagrangeFunctiontozero:

@

Lðw;b;aÞ¼0;@b

nXi¼1

@

Lðw;b;aÞ¼0@w

nXi¼1

Thisgivestheconditions:

aiyi¼0

and

w¼

aiyixi

substitutingintoLðw;b;󰀁Þ,theoptimizationproblemcanbeexpressedas:

Max:WðaÞ¼

nXi¼1

nXn󰀃󰀂1X

aiÀaiajyiyjxi;xj

2i¼1j¼1

(6)

subjectto:

ai!0;

i¼1;.........;n

nXi¼1

aiyi¼0

xiwithnonzerovalueof󰀁iarecalledsupportvectors:

yi½hw;xiiþb󰀄¼1

)

ai>0

(7)

Inthiscase󰀁iisamargin)

yi½hw;xiiþb󰀄>1

)

ai¼0

xiarecalledsupportvectors.Or

(8)

)xiarenotconsideredtobesupportvectors

Incaseofanonlinearlyseparabledataset(Figure10),theslackvariable󰀂iisintroducedleadingtoasoftmarginnclassifier:

X12

MinimizekwkþC󰀂isubjectto:

2i¼1

yiðwÁxiþbÞ!1Àzi

8i

(9)

ClassificationwithArtificialNeuralNetworksandSupportVectorMachines421

Figure10.Classificationofnonlinearlyseparabledataset.

TheparameterCdescribesthetrade-offbetweenthemaximalmarginandthecorrectclassification(LipoWang,2005).Theseidealeadtothefollowingdualproblem

Max:WðaÞ¼

nXi¼1

nXn󰀃󰀂1XaiÀaiajyiyjxi;xj

2i¼1j¼1

(10)

subjectto:

C!ai!0;

i¼1;.........;n

nXi¼1

aiyi¼0

Onecannotethatitisthesimilartooptimizationproblemincaseoflinearlyseparable

datawiththeexceptionof󰀁iislimitedwithupperboundC.

Tosolvenon-linearclassificationproblems,thelinearsupportvectormachinesareappliedtohighdimensionalspaces(Figure11).Transformingdataintoahighdimensionalspacecantransformcomplexproblems(withcomplexdecisionsurfaces)intosimplerproblemsthatcanbesolvedlinearclassifiers(NeuroDimension,Inc.).󰀃󰀂with󰀃Thismeans󰀂transformingfromxi;xjtoFðxiÞ;FðxjÞi(KerneltrickKðxi;xjÞ¼󰀃

FðxiÞ;FðxjÞleadingtothefollowingdualproblem:

Max:WðaÞ¼

nXi¼1

nXn

1XaiÀaiajyiyjKðxi;xjÞ

2i¼1j¼1

(11)

422K.M.Almhdi,P.Valigi,V.Gulbinas,R.Westphal&R.Reuter

Figure11.Transformingdatasetfromtheinputspacetothehighdimentionalspace.

ThecommonKernelfunctionsarepolynomialwithdegreed,theRadialBaseFunctionwithwidth'andtheSigmoidwithparameterk

Kðxi;xjÞ¼

󰀂Ádxi;xjþC󰀇󰀈

󰀆21󰀆

Kðxi;xjÞ¼expÀ2󰀆xiÀxj󰀆

2sÀ󰀃󰀂Á

Kðxi;xjÞ¼tanhkxi;xjþY

À󰀃

(12)(13)(14)

AsincaseofANNs,NeuroSolutionswasusedinthiswork(NeuroDimension,Inc.).Inthisprogram,transformingthedatafrominputspacetohighdimensionalspaceisdoneusingaRadialBasisFunction(RBF)networkthatplacesaGaussianateachdatasample(NeuroDimension,Inc.).Thus,thefeaturespacebecomesaslargeasthenumberofsamples(NeuroDimension,Inc.).Figure12showsaSVMthatisusedtoclassifyoilfluorescencespectralsignature.TheSVMwasdividedintotwopartstoimplementtheRBFdimensionalityexpansionandlargemarginclassifier.AsinthecaseofMLP,SVMsusetheconceptofbackpropagationtrainingtotrainthelinearcombinationofguassians.SVMsaremotivatedbytheconceptoftraininganduseonlythoseinputsthatarenearthedecisionsurfacesincetheyprovidethemostinformationabouttheclassification

Figure12.SupportVectorMachineusedtoclassifyoilfluorescence.

ClassificationwithArtificialNeuralNetworksandSupportVectorMachines423

(NeuroDimension,Inc.).ThecomponentsoftheconstructedSVMareexplainedindetailinreference(NeuroDimension,Inc.).

Thetraining,testing,andproductionprocessesofSVMwerecarriedoutbasedonthesamedatasetthathasbeenusedinMLPtoensureanexactcomparisoninthequalityoftheresultsbetweentheMLPandSVM.2.4Classificationofnoisyoilspectra

Meetingthepracticalsituationofnoisyoilfluorescencedatarequiresinvestigatingthenoise’seffectontheoiltypeclassification.Thishasbeenachievedbytwoapproaches.ThefirstistointroducenoisyoilfluorescencedatatobeclassifiedbytheMLPandSVMthathavebeentrainedontheidealoilspectra(non-noisyspectra)measuredinthelaboratoryinordertoseethetrainedMLP’sandSVM’sabilitytoclassifytheminthepresenceofnoise.Thesecondapproachistoexpandthedatasettoincludebothnon-noisyandnoisyspectraldata.Theresultsachievedwithbothapproachesarecompared.2.4.1Useofthenoise-freedataset

Thesamethreenewnon-noisyinputsoilspectrathatwereclassifiedwiththetrainedMLPandSVMhavebeenpresentedonceagaintothesametrainedMLPandSVM,butasnoisyspectra.Forthis,5%and10%randomGaussiannoisewereaddedtothesenewinputspectraandthenpresentedthemtothetrainedMLPandSVM.2.4.2Trainingwithnoisyspectra

ToensurethebestperformanceofthetrainedMLPandSVMtheoriginaldatasetwasexpandedtoinclude,inadditiontotheidealspectra(withoutnoise),thesamespectraafterthe10%randomGaussiannoisewasaddedtothem.I.e.,theoriginaldatasethasbeendoubledtoincludealsonoisyspectrawitharandomGaussiannoiseof10%.Then,thesamethreenewnoisyspectrawerepresentedtothetrainedMLPandSVM.3RESULTSANDDISCUSSION

3.1OilclassificationandidentificationusingtheCRM

Figure13showstheresultsoftheoilclassificationusingtheCRM.Mostoftheoilswithinthecrudeclasshaveshapefluorescencespectralsignaturesclosetoeachother,thereforetheoilswithinthisclassformtheirowngroupwhentheCRMisapplied.Thesameroleisappliedtoheavyoil.Oilswithinthesludgeoilclassesarerandomlydistributedduetotheoilswithinthisclasshavingadifferentshapeoilfluorescencespectrathanoneanother.Itisshownalsothatsomeoilsfromtheheavyandcrudeoilshaveclosespectratoeachother,whichmakesthemmixedwithinthetwoclassificationgroups.Soifthedetectedoilfluorescencecomesinyellowareawillbemostlyexpectedcrudeoil,andifitcomesinredareawillbemostlyexpectedtobeheavyoil,otherwise,itwillbeexpectedtobesludgeoil.Thisclassificationbasedontheassumptionthatthereareonlythreeclassesoftheoil.Onecanexpandthedatasettoincludeothertypesofoil

424K.M.Almhdi,P.Valigi,V.Gulbinas,R.Westphal&R.Reuter

Figure13.OilclassificationusingtheCRM.

Figure14.LearningcurveofthetrainedMLP.

classessuchasbilgeoilandvegetableoiltoincreasetheabilityofsuchmethodtoclassifyoilfluorescencespectra.

3.2OilclassificationandidentificationusingMLP

Figure14showsthelearningcurveofthetrainednetworkwhereonecanseethatameansquarederrorforbothtraining(T)andcrossvalidation(CV)approachzero.Table1showsthetestingresultsofthetrainednetwork.TheoutputofthetestedMLPshowsthattheclosestvalueto1isforthesludgeoilforthefirstinputandheavyoilisthesecondinputandcrudeoilisthethirdinput.ThisresultmeetsthedesiredoutputperformanceandindicatesthetrainedMLPmeetsthedesiredrequirements.Table2showstheidentificationofunknownoilspectrabythetrainednetwork.Theywerethreenewfluorescencespectralsignaturesofoil,whichthetrainedMLPwasunawareof.Theresultshowsthatthefirstoilissludge,thesecondiscrude,andthethirdisheavyoil.Theseidentificationsweremadebasedonwhichclasshadtheclosestvaluetoone.BasedonthedatabasetheresultsachievedbythetrainedMLPare100%correct.

Tables3and4showtheresultsoftheretrainednetwork.Fromtheresultsonecanseethattheclassificationisstillcorrect,buttheaccuracyoftheclassificationisnotthesamealthoughbothnetworksweretrainedbasedonthesamedataset.Thismeansthatoneshouldtrainthenetworkmanytimestoevaluatethebestperformanceofthetrainednetwork.

Table1.TestingresultsofthetrainedMLP.

ClassificationwithArtificialNeuralNetworksandSupportVectorMachines425

Thefollowingresultsarethoseofthere-trainedMLP:

Figure15.Learningcurvethere-trainedMLP.

Table2.TheidentificationofunknownoilspectrabythetrainedMLP.Theusedspectrawerefromsludge,crudeandheavyoil.

Figure16.(a)theactivecostcurveapproacheszero,whichmeansthatclassificationofthedatasethasbeencarriedoutcorrectly.(b)theperformance,showingthemeansquarederror(MSE),thenormalizedmeansquarederror(NMSE),percenterror(%error),Akaike’sinformationcriterion(AIC),andRissanen’sminimumdescriptionlength(MDL)criterion.

processhasnoaffectontheresult.ThismeanstheperformanceoftheSVMisachievedfromthefirsttimetrainingprocess.Thiswillsavetheuser’stime.Theidentificationofanunknownoilfluorescencespectrumis100%correct.Theseidentificationsweremadebasedonwhichclasshadtheclosestvaluetoone,thesamerolewasappliedincaseofMLP.IntermsoftheaccuracywiththecomparisontoMLPonecanseethatbothMLPandSVMshavecorrectlyidentifiedtheunknownspectra.3.4Classificationofnoisyoilspectra3.4.1Useofthenoise-freedataset

Tables7to10showtheidentificationofthenewnoisyoilspectrathatwerepresentedtothetrainedMLPandSVM.TheseresultsshowthatthetrainedMLPandSVMidentifiedthenoisyinputs,althoughtheMLPandSVMweretrainedwiththenoise-freedataset.Againtheidentificationisbasedontheoutputwhosevalueisclosestto1.Thus,thefirstinputsamplewouldbeidentifiedassludgeoil,thesecondoneasacrudeoilandthethirdoneasheavyoil.Thisidentificationiscorrectaccordingtotheusedtypesofoils.

Table6.IdentificationofunknownoilspectrabyaSVM.SamespectraasinTable2.

ClassificationwithArtificialNeuralNetworksandSupportVectorMachines427

Table8.IdentificationofunknownnoisyspectrabythetrainedMLP.SamespectraasinTable2afteradding10%randomGaussiannoise.

Table14.IdentificationofunknownnoisyspectrabythetrainedSVM.SamespectraasinTable2afteradding10%randomGaussiannoise.

REFERENCES

Agatonovic-KustrinS.&R.Beresford,2000.Basicconceptsofartificialneuralnetwork(ANNmodelinganditsapplicationinpharmaceuticalresearch.JournalofPharmaceuticalandBiomedicalAnalysis,22(5):717–727

Boser,B.E.etal.,A,1992.TrainingAlgorithmforOptimalMarginClassifiers,ProceedingsoftheFifthAnnualWorkshoponComputationalLearningTheory5144–152,Pittsburgh.CatalogueofOpticalSpectraofOils,2005(UniversityofOldenburg,InstituteofPhysics,SectionMarinePhysics)

DucTruongPham&LiuXing,1999.NeuralnetworksforIdentification,Predictionandcontrol,IntelligentSystemsLaboratory,SystemsDivision,CardiffSchoolofEngineering,UniversityofWales,Cardiff.CF23TE,UK:1–3

GexiangZhang,WeidongJin&LaizhaoHu,2004.Radaremittersignalrecognitionbasedonsupportvectormachines.In:Control,Automation,RoboticsandVisionConference(ICARCV2004),2:826–831.DOI10.1109/ICARCV.2004.146

GordanM,C.Kotropoulos&I.Pitas,2002.Applicationofsupportvectormachinesclassifierstovisualspeechrecognition.In:ProceedingsoftheInternationalConferenceonImageProcessing,3:III-129-III-132.DOI10.1109/ICIP.2002.10321

HengstermannT.&R.Reuter,1990.Lidarfluorosensingofmineraloilspillsontheseasurface.AppliedOptics,29:3218–3227

Hoge,F.E.&R.N.Swift,1980.Oilfilmthicknessmeasurementusingairbornelaser-inducedwaterRamanbackscatter.AppliedOptics,19:3269–3281

HongweiYang,ChenHe,HongwenZhu&WentaoSong,2000.Predictionofslantpathrainattenuationbasedonartificialneuralnetwork.In:CircuitsandSystems,2000.Proceedings.ISCAS2000Geneva.The2000IEEEInternationalSymposiumonCircuitsandSystems,1,152–155,DOI10.1109/ISCAS.2000.857049

Jervis,B.W.,MRSaatchi,A.Lacey,T.Roberts,E.M.Allen,N.R.Hudson,S.Oke&M.Grimsley,1994.ArtificialneuralnetworkandspectrumanalysismethodsfordetectingbraindiseasesfromtheCNVresponseintheelectroencephalogram.IEEEProceedingsonScience,MeasurementandTechnology,141(6):432–440

Karpicz,R.,A.Dementjev,Z.Kuprionis,S.Pakalnis,R.Westphal,R.Reuter&V.Gulbinas,2005.Laserfluorosensorforoilspotdetection.LithuanianJournalofPhysics,45(3):213–218

Kevin,L.PriddyandPaulE.Keller,2005.ArtificialNeuralNetworks:Anintroduction,SPIE-TheInternationalSocietyforOpticalEngineering,Bellingham,Washington98227-0010USA,2–5,ISBN:0-8194-5987-9

Kim,W.J.,S.H.Chang&B.H.Lee,1993.Applicationofneuralnetworkstosignalpredictioninnuclearpowerplant.IEEETransactionsonNuclearScience,40(5):1337–1341DOI10.1109/23.2347

LipoWang,2005.SupportVectorMachines:TheoryandApplications(Springer-Verlag,BerlinHeidelberg)136–141,SPIN:10984697

MingTian,Song-CanChen,YiZhuang&JiaLiu,2004.Usingstatisticalanalysisandsupportvectormachineclassificationtodetectcomplicatedattacks.,2004.In:Proceedingsof2004InternationalConferenceonMachineLearningandCybernetics,5:2747–2752DOI10.1109/ICMLC.2004.1378327

MingZhang&J.Fulcher,1996.Facerecognitionusingartificialneuralnetworkgroup-basedadaptivetolerance(GAT)trees.IEEETransactionsonNeuralNetworks,7(3):555–567DOI10.1109/72.501715

430K.M.Almhdi,P.Valigi,V.Gulbinas,R.Westphal&R.Reuter

NeuroDimension,Inc.(Gainsville,Florida,USA),http://www.neurosolutions.com/downloads/documentation.html

Naguib,R.N.G,H.A.M.Sakim,M.S.Lakshmi,V.Wadehra,T.W.J.Lennard,J.Bhatavdekar&G.V.Sherbet,1999.DNAploidyandcellcycledistributionofbreastcanceraspiratecellsmeasuredbyimagecytometryandanalyzedbyartificialneuralnetworksfortheirprognosticsignificance.IEEETransactionsonInformationTechnologyinBiomedicine,3(1):61–69DOI10.1109/4233.7476

YuchunTang,BoJin,YiSun&Yan-QingZhang,2004.Granularsupportvectormachinesformedicalbinaryclassificationproblems.2004.In:Proceedingsofthe2004IEEESymposiumonComputationalIntelligenceinBioinformaticsandComputationalBiology(CIBCB’04),73–78

ClassificationwithArtificialNeuralNetworksandSupportVectorMachines431