Java NIO通道概览与文件通道【源码笔记】

2020-02-18 16:54:56 浏览数 (1)

目录

一、通道概览

1.概念示意图

2.Channel接口继承关系

二、文件通道使用

1.文件通道类图

2.文件通道示例 三、文件通道开启源码

1.通道开启示例

2.RandomAccessFile创建源码

3.文件打开源码

4.开启通道源码

四、ByteBuffer写入通道源码 五、强制刷盘源码

六、通道重置位点源码

七、读取数据到ByteBuffer源码

1.JDK源码跟踪

2.Native源码跟踪

八、通道关闭源码

1.JDK源码追踪

2.Native方法源码跟踪

九、文件截取源码

十、参考资料

十一、系列文章

代码语言:javascript复制

一、通道概览

1.概念示意图

系统I/O即字节的传输,Channel即传输的通道,文件或网络Socket服务即传输的目的地。

2.Channel接口继承关系

实现Channle的接口

小结:由图可以看出直接继承Channel接口的接口由5个分别为:AsynchronousChannel、NetworkChannel、ReadableByteChannel、WritableByteChannel、InterruptibleChannel。其他接口和类都从这5个接口派生。两个字节操作接口ReadableByteChannel、WritableByteChannel,即:通道只能在字节缓冲区上操作。

二、文件通道使用

1.文件通道类图

2.文件通道示例

以示例方式串下文件通道的基本操作,示例内容为:将字符串写入文件,再读出来打印。

代码语言:javascript复制
File file =new File("/Users/yongliang/mytest/channletst.tmp");

RandomAccessFile randomAccessFile =new RandomAccessFile(file,"rw");
FileChannel fileChannel = randomAccessFile.getChannel();// @1
ByteBuffer buffer = ByteBuffer.allocate(20);
String str ="It's a good day!";

buffer.put(str.getBytes("UTF-8"));
buffer.flip();
while(buffer.hasRemaining()) {
    fileChannel.write(buffer);// @2
}

fileChannel.force(false);// @3

System.out.println("FileChannel current position=" fileChannel.position());

fileChannel.position(0);// @4
ByteBuffer buffer2 = ByteBuffer.allocate(16);
int data = fileChannel.read(buffer2);// @5

System.out.println("length="   data);
System.out.println("content="  new String(buffer2.array()));

fileChannel.close();// @6

@1 开启文件通道

@2 将ByteBuffer数据写入FileChannel

@3 强制刷盘

@4 FileChannel重置到开始位置

@5 从FileChannel中读取数据到ByteBuffer

@6 关闭FileChannel

小结:梳理了FileChannle的继承关系以及通过一个示例说明FileChannle的基本操作。

三、文件通道开启源码

1.通道开启示例

代码语言:javascript复制
RandomAccessFile randomAccessFile = new RandomAccessFile(file,"rw");
FileChannel fileChannel = randomAccessFile.getChannel();

2.RandomAccessFile创建源码

代码位置:java.io.RandomAccessFile

代码语言:javascript复制
public RandomAccessFile(File file, String mode)
    throws FileNotFoundException {
    String name = (file != null ? file.getPath() : null);
    int imode = -1;
    if (mode.equals("r"))
        imode = O_RDONLY;
    else if (mode.startsWith("rw")) {
        imode = O_RDWR;
        rw = true;
        if (mode.length() > 2) {
            if (mode.equals("rws"))
                imode |= O_SYNC;
            else if (mode.equals("rwd"))
                imode |= O_DSYNC;
            else
                imode = -1;
        }
    } // @1

    // ...

    fd = new FileDescriptor(); // @2
    fd.attach(this);
    path = name;
    open(name, imode); // @3
}

@1 r只读模式;rw读写模式;rws模式保证数据同步写入磁盘;rwd模式保证数据和元数据同步写入磁盘。

@2 初始化文件描述符,此时初始值为-1

@3 open调用本地方法打开文件

小结:RandomAccessFile通过Native的open方法打开一个文件。

3.文件打开源码

调用链条

代码语言:javascript复制
1.open(name,imode);
2.open0(Stringname,intmode)
3.Java_java_io_RandomAccessFile_open0
4.fileOpen
5.handleOpen

代码位置:jdk/src/solaris/native/java/io/io_util_md.c

代码语言:javascript复制
Java_java_io_RandomAccessFile_open0(JNIEnv*env,
jobjectthis, jstring path, jint mode)
{
intflags = 0;
if(mode & java_io_RandomAccessFile_O_RDONLY)
flags =O_RDONLY;
elseif (mode & java_io_RandomAccessFile_O_RDWR) {
flags =O_RDWR | O_CREAT;
if(mode & java_io_RandomAccessFile_O_SYNC)
flags|= O_SYNC;
elseif (mode & java_io_RandomAccessFile_O_DSYNC)
flags|= O_DSYNC;
}
fileOpen(env,this, path, raf_fd, flags);
}

代码位置:jdk/src/solaris/native/java/io/io_util_md.c

代码语言:javascript复制
fileOpen(JNIEnv *env, jobjectthis, jstring path, jfieldID fid,int flags)
{
WITH_PLATFORM_STRING(env, path, ps) {
    FD fd;

#if defined(__linux__) || defined(_ALLBSD_SOURCE)
/* Remove trailing slashes, since the kernel won't */
char *p = (char *)ps  strlen(ps) -1;
while ((p > ps) && (*p =='/'))
        *p-- ='';
#endif
    fd = handleOpen(ps, flags,0666);
if (fd !=-1) {
        SET_FD(this, fd, fid);
    }else {
        throwFileNotFoundException(env, path);
    }
} END_PLATFORM_STRING(env, ps);
}

代码位置:jdk/src/solaris/native/java/io/io_util_md.c

代码语言:javascript复制
handleOpen(constchar *path,int oflag,int mode) {
FD fd;
RESTARTABLE(open64(path, oflag, mode), fd);// @1
if (fd !=-1) {
structstat64buf64;
int result;
    RESTARTABLE(fstat64(fd, &buf64), result);
if (result !=-1) {
if (S_ISDIR(buf64.st_mode)) {
            close(fd);
            errno = EISDIR;
            fd =-1;
        }
    }else {
        close(fd);
        fd =-1;
    }
}
return fd;
代码语言:javascript复制
#define RESTARTABLE(_cmd, _result) do { 
do { 
        _result = _cmd; // @2
    }while((_result ==-1) && (errno == EINTR)); 
}while(0)

@1 通过open调用链条跟踪,最后调用open64函数来打开一个文件,并返回文件描述符

@2 将文件描述符赋值

open64函数说明

代码语言:javascript复制
The open64()function,similartotheopen()function,opensafileandreturnsanumbercalledafiledescriptor.
open64()differsfromopen()inthatitautomaticallyopensthefilewiththeO_LARGEFILEflagset.

小结:RandomAccessFile的创建,即通过open64()函数打开一个文件返回文件描述符。

4.开启通道源码

代码语言:javascript复制
publicfinal FileChannel getChannel() {
    synchronized (this) {
if (channel ==null) {
            channel = FileChannelImpl.open(fd, path,true, rw,this);
        }
return channel;
    }
}

public static FileChannelopen(FileDescriptor var0, String var1, boolean var2, boolean var3, Object var4) {
return new FileChannelImpl(var0, var1, var2, var3,false, var4);
}

private FileChannelImpl(FileDescriptor var1, String var2, boolean var3, boolean var4, boolean var5, Object var6) {
this.fd = var1;
this.readable = var3;
this.writable = var4;
this.append = var5;
this.parent = var6;
this.path = var2;
this.nd = new FileDispatcherImpl(var5);
}

小结:开启通道即创建FileChannelImpl实例。

四、ByteBuffer写入通道源码

给予以上fileChannel.write(buffer)示例分析。

代码位置:sun.nio.ch.FileChannelImpl

代码语言:javascript复制
public int write(ByteBuffer var1) throws IOException {
// ...
    synchronized(this.positionLock) {
// ...
try {
this.begin();// @1
// ...
do {
 var3 = IOUtil.write(this.fd, var1, -1L,this.nd); // @2
}while(var3 == -3 &&this.isOpen());
int var5 = IOStatus.normalize(var3);
return var5;
}finally {
}

@1/@3 可中断I/O操作,另文分析

@2 执行IOUtil.write操作

代码位置:jdk/src/solaris/native/sun/nio/fs/UnixNativeDispatcher.c

代码语言:javascript复制
Java_sun_nio_fs_UnixNativeDispatcher_write(JNIEnv* env, jclass this, jint fd,
    jlong address, jint nbytes)
{
    ssize_t n;
    void* bufp = jlong_to_ptr(address);
    RESTARTABLE(write((int)fd, bufp, (size_t)nbytes), n); // @1
    if (n == -1) {
        throwUnixException(env, errno);
    }
    return (jint)n;
}

@1 本地函数执行write方法

pwrite函数说明

代码语言:javascript复制
write函数说明
代码语言:javascript复制

小结:将ByteBuffer写入FileChannel,底层通过pwrite()和write()将字节写入到文件。

五、强制刷盘源码

以下源码给予fileChannel.force()进行跟踪展开。

代码位置:sun.nio.ch.FileDispatcher

代码语言:javascript复制
public void force(boolean var1) throws IOException {
     // ...
     var2 = this.nd.force(this.fd, var1); // @1
     // ...

}

int force(FileDescriptor var1, boolean var2) throws IOException {
   return force0(var1, var2);
}

static native int force0(FileDescriptor var0, boolean var1) throws IOException; // @2

@1 调用FileDispatcherImpl#force强制刷盘

@2 调用native方法force0

代码位置:jdk/src/solaris/native/sun/nio/ch/FileDispatcherImpl.c

代码语言:javascript复制
Java_sun_nio_ch_FileDispatcherImpl_force0(JNIEnv *env, jobject this,
                                          jobject fdo, jboolean md)
{
    jint fd = fdval(env, fdo);
    int result = 0;

    if (md == JNI_FALSE) {
        result = fdatasync(fd); // @1
    } else {
#ifdef _AIX
        int getfl = fcntl(fd, F_GETFL);
        if (getfl >= 0 && (getfl & O_ACCMODE) == O_RDONLY) {
            return 0;
        }
#endif
        result = fsync(fd); // @2
    }
    return handle(env, result, "Force failed");
}

@1 如果fileChannel.force(false)执行fdatasync()函数

@2 如果fileChannel.force(true)执行fsync()函数

fdatasync()函数说明

代码语言:javascript复制
fdatasync() is similar to fsync(), but does not flush modified metadata unless that metadata is needed in order to allow a subsequent data retrieval to be correctly handled.

fsync()函数说明

代码语言:javascript复制
The fsync() function transfers all data for the file indicated by the open file descriptor file_descriptor to the storage device associated with file_descriptor. fsync() does not return until the transfer is complete, or until an error is detected.

小结:FileChannel.force(false)调用Native函数fdatasync()同步刷盘,不写入元数据;FileChannel.force(true)调用Native函数fsync()同步刷盘,同时写入元数据信息;元数据包括修改人、修改时间等信息。

六、通道重置位点源码

给予fileChannel.position(0)进行源码追踪。

代码位置:sun.nio.ch.FileChannelImpl

代码语言:javascript复制
public FileChannel position(long var1) throws IOException {
        this.ensureOpen();
        // ...
        do {
            var4 = this.nd.seek(this.fd, var1);
        } while(var4 == -3L && this.isOpen());
        // ...
    }
}

long seek(FileDescriptor var1, long var2) throws IOException {
    return seek0(var1, var2);
}

static native long seek0(FileDescriptor var0, long var1) throws IOException; // @1

@1 调用Native lseek()函数重置位点。

lseek()函数说明

代码语言:javascript复制
The lseek() function changes the current file offset to a new position in the file. The new position is the given byte offset from the position specified by whence. After you have used lseek() to seek to a new location, the next I/O operation on the file begins at that location.

小结:由以上源码可以看出,fileChannel.position(0)通过Native方法seek0来实现位点重置,底层为lseek()函数重置文件位点。

七、读取数据到ByteBuffer源码

给予fileChannel.read(buffer2)进行源码跟踪

1.JDK源码跟踪

代码位置:sun.nio.ch.FileChannelImpl

代码语言:javascript复制
public int read(ByteBuffer var1) throws IOException {
    // ...
    if (this.isOpen()) {
        do {
            var3 = IOUtil.read(this.fd, var1, -1L, this.nd);
        } while(var3 == -3 && this.isOpen());

        int var12 = IOStatus.normalize(var3);
        return var12;
    }
    // ...
}
代码语言:javascript复制
static int read(FileDescriptor var0, ByteBuffer var1, long var2, NativeDispatcher var4) throws IOException {
    // ...
    try {
        int var6 = readIntoNativeBuffer(var0, var5, var2, var4);
        var5.flip();
        if (var6 > 0) {
            var1.put(var5);
        }

    } finally {
        Util.offerFirstTemporaryDirectBuffer(var5);
    }
    // ...
}
代码语言:javascript复制
private static int readIntoNativeBuffer(FileDescriptor var0, ByteBuffer var1, long var2, NativeDispatcher var4) throws IOException {
    if (var2 != -1L) {
        var9 = var4.pread(var0, ((DirectBuffer)var1).address()   (long)var5, var7, var2);
    } else {
        var9 = var4.read(var0, ((DirectBuffer)var1).address()   (long)var5, var7);
    }

    if (var9 > 0) {
        var1.position(var5   var9);
    }

    return var9;
    }
    }

小结:从JDK源码fileChannel.read最终调用Native的pread和read来读取。

2.Native源码跟踪

代码语言:javascript复制
Java_sun_nio_ch_FileDispatcherImpl_read0(JNIEnv *env, jclass clazz,
                             jobject fdo, jlong address, jint len)
{
    jint fd = fdval(env, fdo);
    void *buf = (void *)jlong_to_ptr(address);

    return convertReturnVal(env, read(fd, buf, len), JNI_TRUE);
}

JNIEXPORT jint JNICALL
Java_sun_nio_ch_FileDispatcherImpl_pread0(JNIEnv *env, jclass clazz, jobject fdo,
                            jlong address, jint len, jlong offset)
{
    jint fd = fdval(env, fdo);
    void *buf = (void *)jlong_to_ptr(address);

    return convertReturnVal(env, pread64(fd, buf, len, offset), JNI_TRUE);
}

read()函数说明

代码语言:javascript复制
read() attempts to read up to count bytes from file descriptor fd
into the buffer starting at buf.

pread64()函数说明

代码语言:javascript复制

小结:分别调用了Native的read()函数和pread64()函数,都是从文件描述符读取数据到ByteBuffer中,pread64()支持大文件读取。

八、通道关闭源码

给予fileChannel.close()进行追踪

1.JDK源码追踪

代码位置:java.nio.channels.spi.AbstractInterruptibleChannel

代码语言:javascript复制
public final void close() throws IOException {
    synchronized (closeLock) {
        if (!open)
            return;
        open = false;
        implCloseChannel();
    }
}
代码语言:javascript复制
protected void implCloseChannel() throws IOException {
    if (this.fileLockTable != null) {
        Iterator var1 = this.fileLockTable.removeAll().iterator();

        while(var1.hasNext()) {
            FileLock var2 = (FileLock)var1.next();
            synchronized(var2) {
                if (var2.isValid()) {
                    this.nd.release(this.fd, var2.position(), var2.size());
                    ((FileLockImpl)var2).invalidate();
                }
            }
        }
    }

    this.threads.signalAndWait();
    if (this.parent != null) {
        ((Closeable)this.parent).close();
    } else {
        this.nd.close(this.fd);
    }

}

void release(FileDescriptor var1, long var2, long var4) throws IOException {
    release0(var1, var2, var4);
}

小结:通道关闭调用最终调用Native方法release0。

2.Native方法源码跟踪

代码语言:javascript复制
Java_sun_nio_ch_FileDispatcherImpl_release0(JNIEnv *env, jobject this,
                                         jobject fdo, jlong pos, jlong size)
{
    jint fd = fdval(env, fdo);
    jint lockResult = 0;
    struct flock64 fl;
    int cmd = F_SETLK64;

    fl.l_whence = SEEK_SET;
    if (size == (jlong)java_lang_Long_MAX_VALUE) {
        fl.l_len = (off64_t)0;
    } else {
        fl.l_len = (off64_t)size;
    }
    fl.l_start = (off64_t)pos;
    fl.l_type = F_UNLCK;
    lockResult = fcntl(fd, cmd, &fl);
    if (lockResult< 0) {
        JNU_ThrowIOExceptionWithLastError(env, "Release failed");
    }
}

fcntl()函数说明

代码语言:javascript复制
The fcntl() function performs various actions on open descriptors, such as obtaining or changing the attributes of a file or socket descriptor.

F_SETLK64

代码语言:javascript复制
Sets or clears a file segment lock for a large file. You must specify a third argument of type struct flock64

小结:FileChannel的关闭操作,通过调用fcntl()函数参数为F_SETLK64,清理该文件上的锁。

九、文件截取源码

给予fileChannel.truncate(10)来进行源码跟踪。

代码语言:javascript复制
public FileChannel truncate(long var1){
     var4 = this.nd.truncate(this.fd, var1);
}

static native int truncate0(FileDescriptor var0, long var1) throws IOException;
代码语言:javascript复制
Java_sun_nio_ch_FileDispatcherImpl_truncate0(JNIEnv *env, jobject this,
                                             jobject fdo, jlong size)
{
    return handle(env,
                  ftruncate64(fdval(env, fdo), size),
                  "Truncation failed");
}
ftruncate64函数说明
代码语言:javascript复制
The truncate() and ftruncate() functions cause the regular file named by path or referenced by fd to be truncated to a size of precisely length bytes.
If the file previously was larger than this size, the extra data is lost. If the file previously was shorter, it is extended, and the extended part reads as null bytes ('').
The file offset is not changed.
If the size changed, then the st_ctime and st_mtime fields (respectively, time of last status change and time of last modification; see stat(2)) for the file are updated, and the set-user-ID and set-group-ID permission bits may be cleared.
With ftruncate(), the file must be open for writing; with truncate(), the file must be writable.

小结:文件的截取通过Native函数ftruncate64来实现,从文件开始位置截取指定的长度。

十、参考资料

参考书籍:《Java NIO》第三章

文章总结:本文梳理了通道接口继承关系,以文件通道FileChannel的示例入手,跟踪每个操作的Native方法,以及给出这些Native方法的调用源码和说明。

十一、系列文章

系统层面I/O【原理笔记】

Java NIO缓存区基本操作【源码笔记】

Java NIO字节缓存区【源码笔记】


「瓜农老梁 学习同行」

0 人点赞