共享存儲(chǔ)系統(tǒng)編程.ppt

國(guó)家高性能計(jì)算中心（合肥）,第十三章 共享存儲(chǔ)系統(tǒng)編程,國(guó)家高性能計(jì)算中心（合肥）,共享存儲(chǔ)系統(tǒng)編程,13.1 ANSI X3H5共享存儲(chǔ)模型 13.2 POSIX 線程模型 13.3 OpenMP模型,國(guó)家高性能計(jì)算中心（合肥）,編程標(biāo)準(zhǔn)的作用,規(guī)定程序的執(zhí)行模型 SPMD, SMP 等 如何表達(dá)并行性 DOACROSS, FORALL, PARALLEL,INDEPENDENT 如何表達(dá)同步 Lock, Barrier, Semaphore, Condition Variables 如何獲得運(yùn)行時(shí)的環(huán)境變量 threadid, num of processes,國(guó)家高性能計(jì)算中心（合肥）,ANSI X3H5共享存儲(chǔ)器模型,Started in the mid-80s with the emergence of shared memory parallel computers with proprietary directive driven programming environments 更早的標(biāo)準(zhǔn)化結(jié)果PCF共享存儲(chǔ)器并行Fortran 1993年制定的概念性編程模型 Language Binding C Fortran 77 Fortran 90,國(guó)家高性能計(jì)算中心（合肥）,并行塊（工作共享構(gòu)造） 并行塊(psections . end psections) 并行循環(huán)(pdo . Endo pdo) 單進(jìn)程(psingle . End psingle) 可嵌套 非共享塊重復(fù)執(zhí)行 隱式路障(nowait)，顯式路障和阻擋操作 共享/私有變量 線程同步 門插銷(latch)：臨界區(qū) 鎖：test,lock,unlock 事件:wait,post,clear 序數(shù)(ordinal):順序,國(guó)家高性能計(jì)算中心（合肥）,X3H5:并行性構(gòu)造,Program main !程序以順序模式開始,此時(shí)只有一個(gè) A !A只由基本線程執(zhí)行，稱為主線程 parallel !轉(zhuǎn)換為并行模式，派生出多個(gè)子線程（一個(gè)組） B !B為每個(gè)組員所復(fù)制 psections !并行塊開始 section C !一個(gè)組員執(zhí)行C section D !一個(gè)組員執(zhí)行D end psections !等待C和D都結(jié)束 psingle !暫時(shí)轉(zhuǎn)換成順序模式 E !已由一個(gè)組員執(zhí)行 end psingle !轉(zhuǎn)回并行模式 pdo i=1,6 !pdo構(gòu)造開始 F(i) !組員共享F的六次迭代 end pdo no wait !無隱式路障同步 G !更多的復(fù)制代碼 end parallel !轉(zhuǎn)為順序模式 H !初始化進(jìn)程單獨(dú)執(zhí)行H . !可能有更多的并行構(gòu)造 End,國(guó)家高性能計(jì)算中心（合肥）,線程,隱式路障同步,P,Q,R,A,B,C,E,F(1:2),G,H,G,G,F(3:4),F(5:6),D,B,B,隱式路障同步,隱式路障同步,無隱式路障同步,隱式路障同步,國(guó)家高性能計(jì)算中心（合肥）,共享存儲(chǔ)系統(tǒng)編程,13.1 ANSI X3H5共享存儲(chǔ)模型 13.2 POSIX 線程模型 13.3 OpenMP模型,國(guó)家高性能計(jì)算中心（合肥）,POSIX線程模型,IEEE/ANSI標(biāo)準(zhǔn)IEEE POSIX 1003.1c-1995線程標(biāo)準(zhǔn)Unix/NT操作系統(tǒng)層上的，SMP Chorus, Topaz, Mach Cthreads Win32 Thread GetThreadHandle,SetThreadPriority,SuspendThread,ResumeThread TLS(線程局部存儲(chǔ))TlsAlloc, TlsSetValue LinuxThreads:_clone and sys_clone 用戶線程和內(nèi)核線程(LWP)(一到一，一到多，多到多),What Are Threads?,General-purpose solution for managing concurrency. Multiple independent execution streams. Shared state. Preemptive scheduling. Synchronization (e.g. locks, conditions).,Shared state (memory, files, etc.),Threads,國(guó)家高性能計(jì)算中心（合肥）,線程共享相同的內(nèi)存空間。 與標(biāo)準(zhǔn) fork() 相比，線程帶來的開銷很小。內(nèi)核無需單獨(dú)復(fù)制進(jìn)程的內(nèi)存空間或文件描述符等等。這就節(jié)省了大量的 CPU 時(shí)間。 和進(jìn)程一樣，線程將利用多 CPU。如果軟件是針對(duì)多處理器系統(tǒng)設(shè)計(jì)的，計(jì)算密集型應(yīng)用。 支持內(nèi)存共享無需使用繁瑣的 IPC 和其它復(fù)雜的通信機(jī)制。 Linux _clone不可移植，Pthread可移植。 POSIX 線程標(biāo)準(zhǔn)不記錄任何“家族”信息。無父無子。如果要等待一個(gè)線程終止，就必須將線程的 tid 傳遞給 pthread_join()。線程庫(kù)無法為您斷定 tid。,國(guó)家高性能計(jì)算中心（合肥）,POSIX Threads: Basics and Examples by Uday Kamath /abw/parallel/pthreads/pthreads.html POSIX 線程詳解: 一種支持內(nèi)存共享的簡(jiǎn)單和快捷的工具 by Daniel Robbins /developerWorks/linux/thread/posix_thread1/index.shtml,國(guó)家高性能計(jì)算中心（合肥）,國(guó)家高性能計(jì)算中心（合肥）,線程調(diào)用線程管理,POSIX Solaris 2 pthread_create thr_create pthread_exit thr_exit pthread_kill thr_kill pthread_join thr_join pthread_self thr_self,國(guó)家高性能計(jì)算中心（合肥）,線程調(diào)用線程同步和互斥,POSIX Solaris 2 pthread_mutex_init mutex_init pthread_ mutex_destroy mutex_destroy pthread_ mutex_lock mutex_lock pthread_ mutex_trylock mutex_trylock pthread_ mutex_unlock mutex_unlock pthread_cond_init pthread_cond_destroy pthread_cond_wait pthread_cond_timedwait pthread_cond_signal pthread_cond_broadcast,國(guó)家高性能計(jì)算中心（合肥）,Pthreads實(shí)現(xiàn)計(jì)算的實(shí)例 1,國(guó)家高性能計(jì)算中心（合肥）,Pthreads實(shí)現(xiàn)計(jì)算的實(shí)例 2,國(guó)家高性能計(jì)算中心（合肥）,對(duì)生產(chǎn)者驅(qū)動(dòng)的有界緩沖區(qū)問題的Pthread條件變量解,void *producer(void *arg1) int i; for (i=1;i=SUMSIZE;i+) pthread_mutex_lock(,void *consumer(void *arg2) int i,myitem; for (;) pthread_mutex_lock( ,國(guó)家高性能計(jì)算中心（合肥）,共享存儲(chǔ)系統(tǒng)編程,13.1 ANSI X3H5共享存儲(chǔ)模型 13.2 POSIX 線程模型 13.3 OpenMP模型,國(guó)家高性能計(jì)算中心（合肥）,The History of OpenMP What is directive/pragma? Directive-based general purpose parallel programming API with emphasis on the ability to parallelize existing serial programs Why a new standard? Whos Involved? Parallelism model and basic directives Fortran77, Fortran90 C, C+,OpenMP標(biāo)準(zhǔn),國(guó)家高性能計(jì)算中心（合肥）,The History of OpenMP,A key intermediate step was X3H5 in the late 80s. An official standards effort to agree on a parallel dialect of Fortran for shared memory computers. The X3H5 effort failed. It was too big and too late. OpenMP is born: In 1996 a group formed to create an industry standard set of directives for SMP programming This group called itself the OpenMP Architecture Review Board(the ARB) who takes care of OpenMP,國(guó)家高性能計(jì)算中心（合肥）,The History of OpenMP(cont.),The ARB has released the following specifications: OpenMP 1.0 for Fortran, Nov. 1997 OpenMP 1.0 for C/C+, Nov. 1998 OpenMP Fortran Interpretations, Spring 1999 OpenMP 2.0(soon) OpenMP is an evolving standard. Send comments over the feedback link on the OpenMP web site(),國(guó)家高性能計(jì)算中心（合肥）,為什么要建立新標(biāo)準(zhǔn)?,ANSI X3H5, 1994 時(shí)機(jī)不好, 分布式機(jī)器流行 只支持循環(huán)級(jí)并行性，粒度太細(xì) Pthreads(IEEE Posix 1003.4a ) 是為低端(low end)的共享機(jī)器(如SMP)的標(biāo)準(zhǔn) 對(duì)FORTRAN的支持不夠 適合任務(wù)并行, 而不適合數(shù)據(jù)并行 MPI 消息傳遞的編程標(biāo)準(zhǔn), 對(duì)程序員要求高 HPF 主要用于分布式存儲(chǔ)機(jī)器 大量已有的科學(xué)應(yīng)用程序需要很好地被繼承和移植,國(guó)家高性能計(jì)算中心（合肥）,In a Nutshell,A set of directives(library routines, and environment variables) used to annotate a sequential program to indicate how it should be executed in parallel繼承X3H5的許多概念 Portable, Simple and Scalable Shared Memory Multiprocessing API not a new language not automatic parallelization extend base languages: Fortran77, Fortran90, C and C+ Multi-vendor Support, for both UNIX and NT Standardizes Fine Grained(Loop) Parallelism, also Supports Coarse Grained Algorithms,國(guó)家高性能計(jì)算中心（合肥）,OpenMP是什么？,一組編譯制導(dǎo)語(yǔ)句和可調(diào)用的運(yùn)行(run-time)庫(kù)函數(shù), 擴(kuò)充到基本語(yǔ)言中用來表達(dá)程序中的并行性 編譯制導(dǎo)語(yǔ)句包括: 在串行程序中加入下列結(jié)構(gòu) SPMD(Single Program Multiple Data) constructs work-sharing constructs synchronization constructs data environment constructs 運(yùn)行庫(kù)函數(shù)包括: execution environment routines lock routines 另外, 在FORTRAN標(biāo)準(zhǔn)中, 還包括對(duì)環(huán)境變量的描述,國(guó)家高性能計(jì)算中心（合肥）,國(guó)家高性能計(jì)算中心（合肥）,OpenMP當(dāng)前的狀況,1997年10月28日, DEC, IBM, Intel, SGI, 和 Kuch & Associates 等公司的代表們決定制定一種適用于多種硬件平臺(tái)的共享存儲(chǔ)編程的新的工業(yè)應(yīng)用標(biāo)準(zhǔn) 接著, 全球很多的組織和ISV決定支持這一標(biāo)準(zhǔn), 如DOE/ASCI, Livermore Software Technology Corp., Fluent Inc., Absoft Corp. , Ansys Inc. Etc. 目前支持FORTRAN語(yǔ)言, C 和C, 并建有專門的網(wǎng)址 在科研機(jī)構(gòu)中, 也引起了足夠的重視, 被認(rèn)為是21世紀(jì)最受歡迎的并行編程標(biāo)準(zhǔn) OpenMP on NOWs (SC98, Nov. 1998) Integrated OpenMP and MPI on Clusters,國(guó)家高性能計(jì)算中心（合肥）,國(guó)家高性能計(jì)算中心（合肥）,SPMD的程序執(zhí)行模型,P0,P1 P2 . . .,Pn,國(guó)家高性能計(jì)算中心（合肥）,SMP的程序執(zhí)行模型,國(guó)家高性能計(jì)算中心（合肥）,OpenMP的程序執(zhí)行模型,國(guó)家高性能計(jì)算中心（合肥）,Parallel and work sharing directives data environment directives synchronization directives,國(guó)家高性能計(jì)算中心（合肥）,編譯制導(dǎo)語(yǔ)句(1),Work-sharing constructs 將結(jié)構(gòu)內(nèi)的任務(wù)分配到處理機(jī)中, 必須動(dòng)態(tài)地放在Parallel region construct 中, 進(jìn)入這種結(jié)構(gòu)之前并不隱含BARRIER操作 DO(最常用) 有SCHEDULE選項(xiàng), 可以指定采用什么調(diào)度算法 SECTIONS(可以流水線執(zhí)行之) SINGLE(只有一個(gè)處理機(jī)執(zhí)行之),國(guó)家高性能計(jì)算中心（合肥）,Parallel Region: parallel, end parallel Work Sharing: do, sections, single(parallel do, nowait) Fork-Join model of parallel execution(static,dynamic,orphaned),Parallel Region and Work Sharing Directives,國(guó)家高性能計(jì)算中心（合肥）,編譯制導(dǎo)語(yǔ)句(2),指令格式 固定形式 !$OMP 自由形式 !$OMP, *$OMP, C$OMP Parallel Region Construct !$OMP Parallel clause, clause . . . Do I = 1, 20 A(I) = A(I) + B(I) !$OMP End Parallel (隱含BARRIER操作) 其中Clause可以為:PRIVATE(list), SHARED(list),COPYIN(list), FIRSTPRIVATE(list), DEFAULT(PRIVATE|SHARED|NONE), REDUCTION(operation|intrinsic:list), IF(logical_expression),國(guó)家高性能計(jì)算中心（合肥）,DO編譯制導(dǎo)語(yǔ)句,!$OMP DO clause, clause . . . do_loop !$OMP END DO NOWAIT 例子: !$OMP PARALLEL DO DO I = 2, N B(I) = ( A(I) + A(I-1) /2.0 ENDDO !$OMP END DO NOWAIT !$OMP END PARALLEL,國(guó)家高性能計(jì)算中心（合肥）,SECTIONS 編譯制導(dǎo)語(yǔ)句,!$OMP SECTIONS !$OMP SECTION block1 !$OMP SECTION block2 !$OMP SECTION block3 !$OMP END SECTIONS,國(guó)家高性能計(jì)算中心（合肥）,編譯制導(dǎo)語(yǔ)句(3),Data environment constructs THREADPRIVATE Data scope attribute clauses PRIVATE SHARED DEFAULT FIRSTPRIVATE LASTPRIVATE REDUCTION COPYIN,國(guó)家高性能計(jì)算中心（合肥）,Data Scope attribute clauses: Private, Shared, Default, Firstprivate, Lastprivate, Reduction and Copyin/Copyout(value undefined entering/exiting parallel region) Threadprivate directives:Private to a thread but global within the thread(SMP) Fortran:COMMON blocks / C:file scope and static variables,Data Environment Directives,國(guó)家高性能計(jì)算中心（合肥）,編譯制導(dǎo)語(yǔ)句(4),Synchronization constructs MASTER CRITICAL BARRIER ATOMIC FLUSH ORDERED,國(guó)家高性能計(jì)算中心（合肥）,例子(ORDERED),規(guī)定了各個(gè)線程執(zhí)行的順序 !$OMP PARALLEL !$OMP DO ORDERED SCHEDULE(DYNAMIC) DO I = LowBound, UpBound, Step CALL WORK(I) END DO !$OMP END PARALLEL SUBROUTINE WORK(K) !$OMP ORDERED WRITE(*,*) K !$OMP END ORDERED END,國(guó)家高性能計(jì)算中心（合肥）,Synchronization Directives,ma