我们再次回顾下上一篇文章 Android图形显示系统5 图像缓冲区(上) 描述的图像缓冲区。
１．Android 图形缓冲区由哪些部分组成
Android 的图形缓冲区由 Surface，BufferQueue，Layer，GraphicBuffer 四部分组成。BufferQueue 中的 slots 数组最多能存储 64 个 GraphicBuffer，而 GraphicBuffer 就是真正被分配内存，并能存储图形数据的缓冲区
２．Surface 和 Layer 分别是什么
Surface 是提供给图形生产者控制缓冲区的类，Surface 持有 GraphicBufferProducer，简称 gbp，gbp 专门用来创建或获取可用的 GraphicBuffer 以及提交绘制后的 GraphicBuffer
Layer 是提供给图像消费者获取缓冲区的类，Layer 持有 GraphicBufferConsumer，简称 gbc，通过 gbc 用来获取和释放 GraphicBuffer
３．他们是怎么关联起来的
Surface 和 Layer 通过 BufferQueue 关联起来，Surface 持有 BufferQueue 中的 gbp，Layer 持有 BufferQueue 中的 gbc，gbp 和 gbc 的 GraphicBuffer 都存储在 GraphicBufferCore 的 slots 数组中
４．如何创建、获取以及使用缓冲区
Surface 通过调用 gbp 的 dequeue 函数获取 GraphicBuffer，调用 queue 函数提交使用完毕的 GraphicBuffer
Layer 通过调用 gbc 的 acquire 函数获取有数据的 GraphicBuffer，调用 release 释放 GraphicBuffer
总结：当我们想要绘制图像时，需要创建 Surface 和 Layer，图像生产者如 Skia，OpenGL 通过 Surface 调用 dequeue 函数获取一块缓冲区 GraphicBuffer，有了这块缓冲区，就可以在缓冲区上绘制图像了，当绘制完毕后，通过 Surface 调用 queue 函数，将 GraphicBuffer 归还到 BufferQueue。之后，Vsync 通知图像消费者 SurfaceFlinger 调用 Layer 的 acquire 函数，获取绘制好内容的 GraphicBuffer 进行合成与处理，处理完毕后通过 release 函数释放这块缓冲区。
接上一篇文章，我们继续分析 BufferQueue 的创建。
在前面分析 Layer 的创建时，有提到会在 Layer 的初始化函数 onFirstRef 中会创建 BufferQueue，onFirstRef 函数实现如下
/frameworks/native/services/surfaceflinger/Layer.cpp
void Layer::onFirstRef() {
// Creates a custom BufferQueue for SurfaceFlingerConsumer to use
sp<IGraphicBufferProducer> producer;
sp<IGraphicBufferConsumer> consumer;
BufferQueue::createBufferQueue(&producer, &consumer, true);
mProducer = new MonitoredProducer(producer, mFlinger, this);
mSurfaceFlingerConsumer =
new SurfaceFlingerConsumer(consumer, mTextureName, this);
mSurfaceFlingerConsumer->setConsumerUsageBits(getEffectiveUsage(0));
mSurfaceFlingerConsumer->setContentsChangedListener(this);
mSurfaceFlingerConsumer->setName(mName);
if (mFlinger->isLayerTripleBufferingDisabled()) {
mProducer->setMaxDequeuedBufferCount(2);
}
const sp<const DisplayDevice> hw(mFlinger->getDefaultDisplayDevice());
updateTransformHint(hw);
}
onFirstRef 函数中关键的一步就是调用 createBufferQueue 创建 BufferQueue，它的实现如下
/frameworks/native/libs/gui/BufferQueue.cpp
void BufferQueue::createBufferQueue(sp<IGraphicBufferProducer>* outProducer,
sp<IGraphicBufferConsumer>* outConsumer,
bool consumerIsSurfaceFlinger) {
sp<BufferQueueCore> core(new BufferQueueCore());
sp<IGraphicBufferProducer> producer(
new BufferQueueProducer(core, consumerIsSurfaceFlinger));
sp<IGraphicBufferConsumer> consumer(new BufferQueueConsumer(core));
*outProducer = producer;
*outConsumer = consumer;
}
createBufferQueue 做了三件事情
下面详细讲一下这三件事情
在概述里面提到过，BufferQueueCore 拥有一个 slots 数组用来存储 GraphicBuffer，并且最多可能存放 64 个 GraphicBuffer。先看看 BufferQueueCore 的头文件。
/frameworks/native/include/gui/BufferQueueCore.h
namespace android {
class BufferQueueCore : public virtual RefBase {
friend class BufferQueueProducer;
friend class BufferQueueConsumer;
......
private:
......
BufferQueueDefs::SlotsType mSlots;
Fifo mQueue;
std::set<int> mFreeSlots;
std::list<int> mFreeBuffers;
std::list<int> mUnusedSlots;
std::set<int> mActiveBuffers;
......
};
}
#endif
从 BufferQueueCore 的头文件可以看到，BufferQueueCore 除了持有 BufferQueueProducer，BufferQueueConsumer 和 mSlots，还持有很多其他的数据，如 mQueue，mFreeSlots，mFreeBuffers 等。下面介绍一下关键的几个成员变量。
//文件-->/frameworks/native/include/gui/BufferQueueDefs.h
namespace android {
class BufferQueueCore;
namespace BufferQueueDefs {
typedef BufferSlot SlotsType[NUM_BUFFER_SLOTS];
}
}
//文件-->/frameworks/native/include/ui/BufferQueueDefs.h
namespace android {
namespace BufferQueueDefs {
static constexpr int NUM_BUFFER_SLOTS = 64;
}
}
这里介绍一下 BufferSlot，它的数据结构如下。可以看到 BufferSlot 持有一个 GraphicBuffer，以及 BufferState，还有一些其他的数据。
/frameworks/native/include/gui/BufferSlot.h
struct BufferSlot {
BufferSlot()
: mGraphicBuffer(nullptr),
mBufferState(),
……{
}
sp<GraphicBuffer> mGraphicBuffer;
BufferState mBufferState;
......
};
BufferState 又分为五种状态，FREE，DEQUEUED，QUEUED，ACQUIRED，SHARED。了解了 BufferSlot 和 BufferState，我们也就能理解 BufferQueueCore 为什么会有这么多存储 GraphicBuffer 的数据结构，如 slots，mFreeSlots，mFreeBuffers 等等，这些数据结构都是为了给 BufferSlot 分类，以便获取 GraphicBuffer 时更加高效。
了解了 BufferQueueCore，我们接着看 BufferQueueProducer，在前面概述中提到，Surface 会中持 BufferQueueProducer 的 BP 代理。它的构造函数如下。
/frameworks/native/libs/gui/BufferQueueProducer.cpp
BufferQueueProducer::BufferQueueProducer(const sp<BufferQueueCore>& core,
bool consumerIsSurfaceFlinger) :
mCore(core),
mSlots(core->mSlots),
…… {}
BufferQueueProducer 的构造函数是空方法，在初始化成员变量时，会直接将前面创建好的 BufferQueueCore 和 mSlots 赋值到 BufferQueueProducer 的成员变量中。
我们已经知道，图像生产者通过 dequeue 函数来获取一块可用的 GraphicBuffer，并通过 queue 来归还绘制了数据的 Graphicbuffer。
那么我们在 BufferQueueProducer 中看看这两个函数的具体实现。
/frameworks/native/libs/gui/BufferQueueProducer.cpp
status_t BufferQueueProducer::dequeueBuffer(int *outSlot,
sp<android::Fence> *outFence, uint32_t width, uint32_t height,
PixelFormat format, uint32_t usage,
FrameEventHistoryDelta* outTimestamps) {
……
int found = BufferItem::INVALID_BUFFER_SLOT;
while (found == BufferItem::INVALID_BUFFER_SLOT) {
//1，寻找Free状态的BufferSlot
status_t status =
waitForFreeSlotThenRelock(FreeSlotCaller::Dequeue,&found);
}
//2，把找到的slot放到mActiveBuffers中管理
if (mCore->mSharedBufferSlot != found) {
mCore->mActiveBuffers.insert(found);
}
*outSlot = found;
ATRACE_BUFFER_INDEX(found);
attachedByConsumer = mSlots[found].mNeedsReallocation;
mSlots[found].mNeedsReallocation = false;
//修改mBufferState状态为DEQUEUE状态
mSlots[found].mBufferState.dequeue();
//3,如果找到的GraphicBuffer是空的，或者需要重新申请
// 则把设置到BufferSlot的参数全部初始化
if ((buffer == NULL) ||
buffer->needsReallocation(width,
height, format, BQ_LAYER_COUNT, usage))
{
mSlots[found].mAcquireCalled = false;
mSlots[found].mGraphicBuffer = NULL;
mSlots[found].mRequestBufferCalled = false;
mSlots[found].mEglDisplay = EGL_NO_DISPLAY;
mSlots[found].mEglFence = EGL_NO_SYNC_KHR;
mSlots[found].mFence = Fence::NO_FENCE;
mCore->mBufferAge = 0;
mCore->mIsAllocating = true;
returnFlags |= BUFFER_NEEDS_REALLOCATION;
} else {
// We add 1 because that will be the frame number
// when this buffer is queued
mCore->mBufferAge =
mCore->mFrameCounter + 1 - mSlots[found].mFrameNumber;
}
……
if (returnFlags & BUFFER_NEEDS_REALLOCATION) {
//4，如果GraphicBuffer为空，则重新创建GraphicBuffer，并放入对应的slot中
sp<GraphicBuffer> graphicBuffer = new GraphicBuffer(
width, height, format, BQ_LAYER_COUNT, usage,
{mConsumerName.string(), mConsumerName.size()});
status_t error = graphicBuffer->initCheck();
{
if (error == NO_ERROR && !mCore->mIsAbandoned) {
//将新创建的GraphicBuffer放入对应的BufferSLot中
graphicBuffer->setGenerationNumber(
mCore->mGenerationNumber);
mSlots[*outSlot].mGraphicBuffer = graphicBuffer;
}
……
}
}
......
return returnFlags;
}
dequeueBuffer 函数做的事情主要如下
了解了 dequeueBuffer 的流程，在接着看 queueBuffer，queuebuffer 是生产者使用完 GraphicBuffer 后把 GraphicBuffer 放回 BufferQueue，并把 BufferState 修改成 QUEUE 状态的流程，我们看一下源码
/frameworks/native/libs/gui/BufferQueueProducer.cpp
status_t BufferQueueProducer::queueBuffer(int slot,
const QueueBufferInput &input, QueueBufferOutput *output) {
……
BufferItem item;
{
//错误处理
……
mSlots[slot].mFence = acquireFence;
//1，修改mBufferState状态为QUEUE状态
mSlots[slot].mBufferState.queue();
// Increment the frame counter and store a local version of it
// for use outside the lock on mCore->mMutex.
++mCore->mFrameCounter;
currentFrameNumber = mCore->mFrameCounter;
mSlots[slot].mFrameNumber = currentFrameNumber;
//2，BufferItem复制
item.mAcquireCalled = mSlots[slot].mAcquireCalled;
item.mGraphicBuffer = mSlots[slot].mGraphicBuffer;
item.mCrop = crop;
item.mTransform = transform &
~static_cast<uint32_t>(NATIVE_WINDOW_TRANSFORM_INVERSE_DISPLAY);
item.mTransformToDisplayInverse =
(transform & NATIVE_WINDOW_TRANSFORM_INVERSE_DISPLAY) != 0;
item.mScalingMode = static_cast<uint32_t>(scalingMode);
item.mTimestamp = requestedPresentTimestamp;
item.mIsAutoTimestamp = isAutoTimestamp;
item.mDataSpace = dataSpace;
item.mFrameNumber = currentFrameNumber;
item.mSlot = slot;
item.mFence = acquireFence;
item.mFenceTime = acquireFenceTime;
item.mIsDroppable = mCore->mAsyncMode ||
mCore->mDequeueBufferCannotBlock ||
(mCore->mSharedBufferMode && mCore->mSharedBufferSlot == slot);
item.mSurfaceDamage = surfaceDamage;
item.mQueuedBuffer = true;
item.mAutoRefresh = mCore->mSharedBufferMode && mCore->mAutoRefresh;
mStickyTransform = stickyTransform;
output->bufferReplaced = false;
if (mCore->mQueue.empty()) {
// BufferQueueCore的BufferSlot队列为空时，直接push到队尾
mCore->mQueue.push_back(item);
frameAvailableListener = mCore->mConsumerListener;
} else {
// 队列不为空，需要判断一下last BufferItem是否被替换
// 如果可以替换就替换，如果不可以替换就直接把BufferItem放到mQueue尾部
const BufferItem& last = mCore->mQueue.itemAt(
mCore->mQueue.size() - 1);
if (last.mIsDroppable) {
if (!last.mIsStale) {
mSlots[last.mSlot].mBufferState.freeQueued();
if (!mCore->mSharedBufferMode &&
mSlots[last.mSlot].mBufferState.isFree()) {
mSlots[last.mSlot].mBufferState.mShared = false;
}
if (!mSlots[last.mSlot].mBufferState.isShared()) {
mCore->mActiveBuffers.erase(last.mSlot);
mCore->mFreeBuffers.push_back(last.mSlot);
output->bufferReplaced = true;
}
}
// Overwrite the droppable buffer with the incoming one
mCore->mQueue.editItemAt(mCore->mQueue.size() - 1) = item;
frameReplacedListener = mCore->mConsumerListener;
} else {
mCore->mQueue.push_back(item);
frameAvailableListener = mCore->mConsumerListener;
}
}
……
}
return NO_ERROR;
}
queueBuffer 的流程主要做了这两件事情：
了解了 BufferQueueCore 和 BufferQueueProducer，接着看 BufferQueue 的最后一个元素：BufferQueueConsumer，在概览中讲到过，图像消费者 SurfceFlinger 通过 acquire 来获取图像缓冲区，通过 release 来释放该缓冲区。下面就看看 BufferQueueConsumer 中 acquire 和 release 两个操作的具体流程。
先看 acquireBuffer 的过程
/frameworks/native/libs/gui/BufferQueueConsumer.cpp
status_t BufferQueueConsumer::acquireBuffer(BufferItem* outBuffer,
nsecs_t expectedPresent, uint64_t maxFrameNumber) {
int numDroppedBuffers = 0;
sp<IProducerListener> listener;
{
//1,检查acquire的buffer的数量是否超出了限制
……
//2.检查BufferQueueCore中的mQueue队列是否为空
if (mCore->mQueue.empty() && !sharedBufferAvailable) {
return NO_BUFFER_AVAILABLE;
}
//获取BufferQueueCore中的mQueue队列的迭代器
BufferQueueCore::Fifo::iterator front(mCore->mQueue.begin());
int slot = BufferQueueCore::INVALID_BUFFER_SLOT;
if (sharedBufferAvailable && mCore->mQueue.empty()) {
//共享Buffer模式
} else {
//从front获取对应的slot
slot = front->mSlot;
*outBuffer = *front;
}
if (!outBuffer->mIsStale) {
mSlots[slot].mAcquireCalled = true;
if (mCore->mQueue.empty()) {
mSlots[slot].mBufferState.acquireNotInQueue();
} else {
//将BufferState状态改为acquire
mSlots[slot].mBufferState.acquire();
}
mSlots[slot].mFence = Fence::NO_FENCE;
}
if (outBuffer->mAcquireCalled) {
outBuffer->mGraphicBuffer = NULL;
}
//将该Buffer从mQueue中移除
mCore->mQueue.erase(front);
}
//回调
……
return NO_ERROR;
}
acquireBuffer 函数中的逻辑也非常的清晰，主要就是这几件事情
再接着看 releaseBuffer 的流程
/frameworks/native/libs/gui/BufferQueueConsumer.cpp
status_t BufferQueueConsumer::releaseBuffer(int slot, uint64_t frameNumber,
const sp<Fence>& releaseFence, EGLDisplay eglDisplay,
EGLSyncKHR eglFence) {
//slot合法性校验
if (slot < 0 || slot >= BufferQueueDefs::NUM_BUFFER_SLOTS ||
releaseFence == NULL) {
return BAD_VALUE;
}
{
//slot状态校验
if (!mSlots[slot].mBufferState.isAcquired()) {
return BAD_VALUE;
}
......
//将BufferState转为release
mSlots[slot].mBufferState.release();
......
// 将slot放在FreeBuffers中
if (!mSlots[slot].mBufferState.isShared()) {
mCore->mActiveBuffers.erase(slot);
mCore->mFreeBuffers.push_back(slot);
}
}
// 回调
if (listener != NULL) {
listener->onBufferReleased();
}
return NO_ERROR;
}
releaseBuffer 的流程也比较简单，主要是将 BufferState 改为 release 状态，并将该 BufferlSlot 放在FreeBuffers 这个容器中。
GraphicBuffer 是图像缓冲区的最后一个组成部分，也是最基本最重要的一个单元，在前面讲到 BufferQueueProucer 通过 dequeueBuffer 获取 BufferSlot 时，会判断 BufferSlot 中的 GraphicBuffer 是否为空，如果为空就会创建新的 GraphicBuffer。
status_t BufferQueueProducer::dequeueBuffer(int *outSlot,
sp<android::Fence> *outFence, uint32_t width, uint32_t height,
PixelFormat format, uint32_t usage,
FrameEventHistoryDelta* outTimestamps) {
......
if (returnFlags & BUFFER_NEEDS_REALLOCATION) {
//如果GraphicBuffer为空，则重新创建GraphicBuffer，并放入对应的slot中
sp<GraphicBuffer> graphicBuffer = new GraphicBuffer(
width, height, format, BQ_LAYER_COUNT, usage,
{mConsumerName.string(), mConsumerName.size()});
status_t error = graphicBuffer->initCheck();
{
if (error == NO_ERROR && !mCore->mIsAbandoned) {
//将新创建的GraphicBuffer放入对应的BufferSLot中
graphicBuffer->setGenerationNumber(mCore->mGenerationNumber);
mSlots[*outSlot].mGraphicBuffer = graphicBuffer;
}
......
}
}
......
return returnFlags;
}
可以看到在 dequeueBuffer 会创建 GraphicBuffer，GraphicBuffer 是什么呢？我们先看看它的构造函数
/frameworks/native/libs/ui/GraphicBuffer.cpp
GraphicBuffer::GraphicBuffer(uint32_t inWidth, uint32_t inHeight,
PixelFormat inFormat, uint32_t inLayerCount, uint64_t usage,
std::string requestorName)
: GraphicBuffer()
{
mInitCheck = initWithSize(inWidth, inHeight, inFormat, inLayerCount,
usage, std::move(requestorName));
}
status_t GraphicBuffer::initWithSize(uint32_t inWidth, uint32_t inHeight,
PixelFormat inFormat, uint32_t inLayerCount, uint64_t inUsage,
std::string requestorName)
{
GraphicBufferAllocator& allocator = GraphicBufferAllocator::get();
uint32_t outStride = 0;
status_t err = allocator.allocate(inWidth, inHeight, inFormat, inLayerCount,
inUsage, &handle, &outStride, mId,
std::move(requestorName));
if (err == NO_ERROR) {
width = static_cast<int>(inWidth);
height = static_cast<int>(inHeight);
format = inFormat;
layerCount = inLayerCount;
usage = static_cast<int>(inUsage);
stride = static_cast<int>(outStride);
}
return err;
}
GraphicBuffer 的构造函数调用 initWithSize 方法创建指定大小的 Buffer，流程如下
/frameworks/native/libs/ui/GraphicBufferAllocator.cpp
GraphicBufferAllocator::GraphicBufferAllocator()
: mMapper(GraphicBufferMapper::getInstance()),
mAllocator(std::make_unique<Gralloc2::Allocator>(
mMapper.getGrallocMapper()))
{
}
从 GraphicBufferAllocator 的构造函数中可以看到，真正的缓冲区分配器是 Gralloc2::Allocator 模块。由于篇幅原因，在这里就不在深入讲解 Gralloc 了，我们只需要了解 Gralloc 是位于 HAL（硬件抽象层）中的模块，封装了对帧缓冲区的所有访问创建等操作。
我们已经对图像缓冲区有了一定的了解了，包括 Surface，Layer，BufferQueue 的创建和使用也都熟悉了。那么接下来看看图像消费者在 Android 中使用图像缓冲区的场景：软件绘制、硬件加速。
在图像生产者中介绍 Skia 时讲到过，软件绘制的入口函数 drawSoftware 的三步操作。
/frameworks/base/core/java/android/view/ViewRootImpl.java
private boolean drawSoftware(Surface surface, AttachInfo attachInfo, int xoff, int yoff,
boolean scalingRequired, Rect dirty) {
final Canvas canvas;
canvas = mSurface.lockCanvas(dirty);
……
mView.draw(canvas);
……
surface.unlockCanvasAndPost(canvas);
……
return true;
}
第一步会创建或获取图像缓冲区，第三步会提交图像缓冲区。
/frameworks/base/core/java/android/view/Surface.java
public Canvas lockCanvas(Rect inOutDirty)
throws Surface.OutOfResourcesException, IllegalArgumentException {
synchronized (mLock) {
mLockedObject = nativeLockCanvas(mNativeObject, mCanvas, inOutDirty);
return mCanvas;
}
}
lockCanvas 会调用 native 方法 nativeLockCanvas
/frameworks/base/core/jni/android_view_Surface.cpp
static jlong nativeLockCanvas(JNIEnv* env, jclass clazz,
jlong nativeObject, jobject canvasObj, jobject dirtyRectObj) {
sp<Surface> surface(reinterpret_cast<Surface *>(nativeObject));
if (!isSurfaceValid(surface)) {
doThrowIAE(env);
return 0;
}
Rect dirtyRect(Rect::EMPTY_RECT);
Rect* dirtyRectPtr = NULL;
if (dirtyRectObj) {
dirtyRect.left = env->GetIntField(dirtyRectObj, gRectClassInfo.left);
dirtyRect.top = env->GetIntField(dirtyRectObj, gRectClassInfo.top);
dirtyRect.right = env->GetIntField(dirtyRectObj, gRectClassInfo.right);
dirtyRect.bottom = env->GetIntField(dirtyRectObj, gRectClassInfo.bottom);
dirtyRectPtr = &dirtyRect;
}
ANativeWindow_Buffer outBuffer;
//1，获取用来存储图形绘制的buffer
status_t err = surface->lock(&outBuffer, dirtyRectPtr);
SkImageInfo info = SkImageInfo::Make(outBuffer.width, outBuffer.height,
convertPixelFormat(outBuffer.format),
outBuffer.format == PIXEL_FORMAT_RGBX_8888
? kOpaque_SkAlphaType : kPremul_SkAlphaType,
GraphicsJNI::defaultColorSpace());
SkBitmap bitmap;
ssize_t bpr = outBuffer.stride * bytesPerPixel(outBuffer.format);
bitmap.setInfo(info, bpr);
if (outBuffer.width > 0 && outBuffer.height > 0) {
//将上一个buffer里的图形数据复制到当前bitmap中
bitmap.setPixels(outBuffer.bits);
} else {
// be safe with an empty bitmap.
bitmap.setPixels(NULL);
}
//2，创建一个 SkCanvas
Canvas* nativeCanvas = GraphicsJNI::getNativeCanvas(env, canvasObj);
//3，给SKCanvas设置Bitmap
nativeCanvas->setBitmap(bitmap);
//如果指定了脏区，则设定脏区的区域
if (dirtyRectPtr) {
nativeCanvas->clipRect(dirtyRect.left, dirtyRect.top,
dirtyRect.right, dirtyRect.bottom, SkClipOp::kIntersect);
}
if (dirtyRectObj) {
env->SetIntField(dirtyRectObj, gRectClassInfo.left, dirtyRect.left);
env->SetIntField(dirtyRectObj, gRectClassInfo.top, dirtyRect.top);
env->SetIntField(dirtyRectObj, gRectClassInfo.right, dirtyRect.right);
env->SetIntField(dirtyRectObj, gRectClassInfo.bottom, dirtyRect.bottom);
}
sp<Surface> lockedSurface(surface);
lockedSurface->incStrong(&sRefBaseOwner);
return (jlong) lockedSurface.get();
}
nativeLockCanvas 函数中会调用了 surface->lock 函数来获取一个供 SkCanvas 使用的缓冲区，看一下具体实现。
/frameworks/native/libs/gui/Surface.cpp
status_t Surface::lock(
ANativeWindow_Buffer* outBuffer, ARect* inOutDirtyBounds)
{
......
ANativeWindowBuffer* out;
int fenceFd = -1;
//1.调用dequeueBuffer获取图像缓冲区
status_t err = dequeueBuffer(&out, &fenceFd);
if (err == NO_ERROR) {
sp<GraphicBuffer> backBuffer(GraphicBuffer::getSelf(out));
const Rect bounds(backBuffer->width, backBuffer->height);
Region newDirtyRegion;
if (inOutDirtyBounds) {
newDirtyRegion.set(static_cast<Rect const&>(*inOutDirtyBounds));
newDirtyRegion.andSelf(bounds);
} else {
newDirtyRegion.set(bounds);
}
const sp<GraphicBuffer>& frontBuffer(mPostedBuffer);
const bool canCopyBack = (frontBuffer != 0 &&
backBuffer->width == frontBuffer->width &&
backBuffer->height == frontBuffer->height &&
backBuffer->format == frontBuffer->format);
if (canCopyBack) {
// 2，如果前一帧的缓冲区大小和格式和当前一样，则将前一帧的缓冲区数据拷贝
// 到当前获取的缓冲区来，这样能复用数据，节约性能。
const Region copyback(mDirtyRegion.subtract(newDirtyRegion));
if (!copyback.isEmpty()) {
copyBlt(backBuffer, frontBuffer, copyback, &fenceFd);
}
} else {
……
}
……
void* vaddr;
status_t res = backBuffer->lockAsync(
GRALLOC_USAGE_SW_READ_OFTEN | GRALLOC_USAGE_SW_WRITE_OFTEN,
newDirtyRegion.bounds(), &vaddr, fenceFd);
……
}
return err;
}
我们直接看 dequeueBuffer 是如何获取缓冲区的
/frameworks/native/libs/gui/Surface.cpp
int Surface::dequeueBuffer(android_native_buffer_t** buffer, int* fenceFd) {
……
//创建缓冲区
status_t result = mGraphicBufferProducer->dequeueBuffer(&buf, &fence,
reqWidth, reqHeight, reqFormat, reqUsage,
enableFrameTimestamps ? &frameTimestamps : nullptr);
if ((result & IGraphicBufferProducer::BUFFER_NEEDS_REALLOCATION) || gbuf == nullptr) {
//映射缓冲区
result = mGraphicBufferProducer->requestBuffer(buf, &gbuf);
}
return OK;
}
dequeueBuffer 关键的步骤只有两步
dequeueBuffer 已经讲过了，就不重复讲了，我们看一下 requestBuffer，当我们获取了缓冲区内存后，图像生产者和图像消费者都需要将缓冲区映射到他们的进程中才能使用。
/frameworks/native/libs/gui/IGraphicBufferProducer.cpp
virtual status_t requestBuffer(int bufferIdx, sp<GraphicBuffer>* buf) {
Parcel data, reply;
data.writeInterfaceToken(IGraphicBufferProducer::getInterfaceDescriptor());
data.writeInt32(bufferIdx);
status_t result =remote()->transact(REQUEST_BUFFER, data, &reply);
if (result != NO_ERROR) {
return result;
}
bool nonNull = reply.readInt32();
if (nonNull) {
//在用户进程创建空的GraphicBuffer
*buf = new GraphicBuffer();
//通过共享内存的方式，映射GraphicBufferProducer中的GraphicBuffer
result = reply.read(**buf);
if(result != NO_ERROR) {
(*buf).clear();
return result;
}
}
result = reply.readInt32();
return result;
}
可以看到，requestBuffer 函数中创建了 GraphicBuffer 对象，这个对应位于用户进程，并且没有申请内存。接着调用了 Parcel 的 read 的方法，将 GraphicBufferProducer 中的 GraphicBuffer 映射过来。共享内存是最高效也是传输数据量最大的 IPC 通信机制。
接着看软绘是如何提交缓冲区，它的实现在 nativeUnlockCanvasAndPost 这个 native 函数中
/frameworks/base/core/jni/android_view_Surface.cpp
static void nativeUnlockCanvasAndPost(JNIEnv* env, jclass clazz,
jlong nativeObject, jobject canvasObj) {
sp<Surface> surface(reinterpret_cast<Surface *>(nativeObject));
if (!isSurfaceValid(surface)) {
return;
}
// detach the canvas from the surface
Canvas* nativeCanvas = GraphicsJNI::getNativeCanvas(env, canvasObj);
nativeCanvas->setBitmap(SkBitmap());
// unlock surface
status_t err = surface->unlockAndPost();
if (err < 0) {
doThrowIAE(env);
}
}
nativeUnlockCanvasAndPost 调用了 surface 的 unlockAndPost 函数
/frameworks/native/libs/gui/Surface.cpp
status_t Surface::unlockAndPost()
{
if (mLockedBuffer == 0) {
return INVALID_OPERATION;
}
int fd = -1;
status_t err = mLockedBuffer->unlockAsync(&fd);
//图像缓冲区的入队操作
err = queueBuffer(mLockedBuffer.get(), fd);
mPostedBuffer = mLockedBuffer;
mLockedBuffer = 0;
return err;
}
在着看 queueBuffer 的实现
/frameworks/native/libs/gui/Surface.cpp
int Surface::queueBuffer(android_native_buffer_t* buffer, int fenceFd) {
Mutex::Autolock lock(mMutex);
int64_t timestamp;
bool isAutoTimestamp = false;
//1，获取Buffer在slots中的index
int i = getSlotFromBufferLocked(buffer);
if (i < 0) {
if (fenceFd >= 0) {
close(fenceFd);
}
return i;
}
if (mSharedBufferSlot == i && mSharedBufferHasBeenQueued) {
if (fenceFd >= 0) {
close(fenceFd);
}
return OK;
}
......
//2,调用GraphicBufferProducer的queueBuffer函数，进行入队操作
status_t err = mGraphicBufferProducer->queueBuffer(i, input, &output);
mLastQueueDuration = systemTime() - now;
if (err != OK) {
ALOGE("queueBuffer: error queuing buffer to SurfaceTexture, %d", err);
}
……
return err;
}
可以看到 surface.unlockCanvasAndPost 最终也是通过 queueBuffer 函数来提交缓冲区的。
了解了软件绘制如何使用缓冲区，我们在看看硬件加速中是如何使用缓冲区的，硬件加速的绘制入口在 CanvasContext 的 draw 函数中，对硬件绘制流程不熟的，可以看图像生产者中使用 OpenGL 进行硬件绘制的部分。
/frameworks/base/libs/hwui/renderthread/CanvasContext.cpp
void CanvasContext::draw() {
SkRect dirty;
mDamageAccumulator.finish(&dirty);
mCurrentFrameInfo->markIssueDrawCommandsStart();
//1，获取缓冲区
Frame frame = mRenderPipeline->getFrame();
SkRect windowDirty = computeDirtyRect(frame, &dirty);
//2，调用OpenGL的draw函数进行绘制
bool drew = mRenderPipeline->draw(frame, windowDirty, dirty, mLightGeometry, &mLayerUpdateQueue,
mContentDrawBounds, mOpaque, mLightInfo, mRenderNodes, &(profiler()));
waitOnFences();
bool requireSwap = false;
//3，提交缓冲区
bool didSwap = mRenderPipeline->swapBuffers(frame, drew, windowDirty, mCurrentFrameInfo,
&requireSwap);
mIsDirty = false;
......
}
硬件加速和软件绘制的流程都是分为三步
我们先看第一步：获取缓冲区。它的是在 RenderPipeline 的 getFrame 函数中，这里的 RenderPipeline 以 OpenGLPipeline 为例讲解。
/frameworks/base/libs/hwui/renderthread/OpenGLPipeline.cpp
Frame OpenGLPipeline::getFrame() {
return mEglManager.beginFrame(mEglSurface);
}
getFrame 调用了 EglManager 的 beginFrame 函数
Frame EglManager::beginFrame(EGLSurface surface) {
makeCurrent(surface);
Frame frame;
frame.mSurface = surface;
eglQuerySurface(mEglDisplay, surface, EGL_WIDTH, &frame.mWidth);
eglQuerySurface(mEglDisplay, surface, EGL_HEIGHT, &frame.mHeight);
frame.mBufferAge = queryBufferAge(surface);
eglBeginFrame(mEglDisplay, surface);
return frame;
}
beginFrame 中的流程有这几步
关键函数是 makeCurrent，直接看它的实现
/frameworks/base/libs/hwui/renderthread/EglManager.cpp
bool EglManager::makeCurrent(EGLSurface surface, EGLint* errOut) {
if (!eglMakeCurrent(mEglDisplay, surface, surface, mEglContext)) {
if (errOut) {
*errOut = eglGetError();
} else {
}
}
mCurrentSurface = surface;
if (Properties::disableVsync) {
eglSwapInterval(mEglDisplay, 0);
}
return true;
}
makeCurrent 实际调用的是 eglMakeCurrent，这部分的实现在 EGL 里面
/frameworks/native/opengl/libs/EGL/egl.cpp
EGLBoolean eglMakeCurrent( EGLDisplay dpy, EGLSurface draw,
EGLSurface read, EGLContext ctx)
{
//draw和read和合法校验
……
if (ctx == EGL_NO_CONTEXT) {
current_ctx = (EGLContext)getGlThreadSpecific();
} else {
egl_context_t* c = egl_context_t::context(ctx);
//将EGLSurface转换成egl_surface_t
egl_surface_t* d = (egl_surface_t*)draw;
egl_surface_t* r = (egl_surface_t*)read;
}
ogles_context_t* gl = (ogles_context_t*)ctx;
if (makeCurrent(gl) == 0) {
if (ctx) {
egl_context_t* c = egl_context_t::context(ctx);
egl_surface_t* d = (egl_surface_t*)draw;
egl_surface_t* r = (egl_surface_t*)read;
if (c->draw) {
egl_surface_t* s = reinterpret_cast<egl_surface_t*>(c->draw);
s->disconnect();
s->ctx = EGL_NO_CONTEXT;
if (s->zombie)
delete s;
}
if (c->read) {
}
c->draw = draw;
c->read = read;
if (c->flags & egl_context_t::NEVER_CURRENT) {
c->flags &= ~egl_context_t::NEVER_CURRENT;
GLint w = 0;
GLint h = 0;
if (draw) {
w = d->getWidth();
h = d->getHeight();
}
ogles_surfaceport(gl, 0, 0);
ogles_viewport(gl, 0, 0, w, h);
ogles_scissor(gl, 0, 0, w, h);
}
if (d) {
//1，调用egl_surface_t的connect函数创建缓冲区
if (d->connect() == EGL_FALSE) {
return EGL_FALSE;
}
d->ctx = ctx;
d->bindDrawSurface(gl);
}
if (r) {
r->ctx = ctx;
r->bindReadSurface(gl);
}
} else {
//异常处理
……
}
return EGL_TRUE;
}
return setError(EGL_BAD_ACCESS, EGL_FALSE);
}
eglMakeCurrent 函数中我们只关心一件事情：执行 d->connect() 创建缓冲区
/frameworks/native/opengl/libs/EGL/egl.cpp
EGLBoolean egl_window_surface_v2_t::connect()
{
//获取图像缓冲区
if (nativeWindow->dequeueBuffer(nativeWindow, &buffer,
&fenceFd) != NO_ERROR) {
return setError(EGL_BAD_ALLOC, EGL_FALSE);
}
……
// 绑定缓冲区
if (lock(buffer, GRALLOC_USAGE_SW_READ_OFTEN |
GRALLOC_USAGE_SW_WRITE_OFTEN, &bits) != NO_ERROR) {
return setError(EGL_BAD_ACCESS, EGL_FALSE);
}
return EGL_TRUE;
}
可以看到 connect 函数中最终依然是调用了nativeWindow 的 dequeueBuffer 函数，前面提到过 Surface 继承自 nativeWindow，所以这里实际就是调用了 Surface 的 dequeueBuffer 函数，这个函数的实现在前面详细讲了，这儿就不再讲了。但是这里和软件绘制有点区别的地方在于缓冲区的绑定，软件绘制中是调用的 requestBuffer 进行共享内存的绑定，这里是调用 lock 函数，它的实现如下
/frameworks/native/opengl/libs/EGL/egl.cpp
status_t egl_window_surface_v2_t::lock(
ANativeWindowBuffer* buf, int usage, void** vaddr)
{
auto& mapper = GraphicBufferMapper::get();
return mapper.lock(buf->handle, usage,
android::Rect(buf->width, buf->height), vaddr);
}
lock 函数里面其实调用 GraphicBufferMapper 进行了一个内存绑定的操作，GraphicBufferMapper 属于 Galloc 模块，它是单例的，在 requesetBuffer 函数中进行内存共享绑定操作时，也会用到这个 GraphicBufferMapper，他们是同一个 GraphicBufferMapper。
我们接着看硬件加速是如何提交缓冲区的，它的实现在 swapBuffers 函数中。
/frameworks/native/opengl/libagl/egl.cpp
EGLBoolean egl_window_surface_v2_t::swapBuffers()
{
……
unlock(buffer);
previousBuffer = buffer;
//1，提交缓冲区
nativeWindow->queueBuffer(nativeWindow, buffer, -1);
buffer = 0;
int fenceFd = -1;
//2，获取新的缓冲区
if (nativeWindow->dequeueBuffer(nativeWindow, &buffer, &fenceFd) == NO_ERROR) {
……
} else {
return setError(EGL_BAD_CURRENT_SURFACE, EGL_FALSE);
}
return EGL_TRUE;
}
swapBuffers 中的逻辑很清晰，主要就两步操作：
关于缓冲区的使用实例这里就讲完了，可以看到，不管是软件绘制，还是硬件加速，使用的方式都是一致的，都为下面几步：
这些步骤是通用的流程，不仅适用在软件绘制或者硬件绘制，还包括 webview，或者是 Flutter，或者我们自己开发渲染引擎，都需要使用同样的步骤。
下面接着看图像消费使用缓冲区的实例。
在 Android图形显示系统2 图像消费者 这一篇中讲到，SurfaceFlinger 会在 onMessageReceived 收到 VSync 的通知
/frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp
void SurfaceFlinger::onMessageReceived(int32_t what) {
ATRACE_CALL();
switch (what) {
case MessageQueue::INVALIDATE: {
bool frameMissed = !mHadClientComposition &&
mPreviousPresentFence != Fence::NO_FENCE &&
(mPreviousPresentFence->getSignalTime() ==
Fence::SIGNAL_TIME_PENDING);
ATRACE_INT("FrameMissed", static_cast<int>(frameMissed));
if (mPropagateBackpressure && frameMissed) {
ALOGD("Backpressure trigger, skipping transaction & refresh!");
//如果掉帧则请求下一次VSync，跳过这一次请求
signalLayerUpdate();
break;
}
//更新VR模式的Flinger
updateVrFlinger();
bool refreshNeeded = handleMessageTransaction();
//获取缓冲区
refreshNeeded |= handleMessageInvalidate();
refreshNeeded |= mRepaintEverything;
if (refreshNeeded) {
//判断是否要做刷新
signalRefresh();
}
break;
}
case MessageQueue::REFRESH: {
handleMessageRefresh();
break;
}
}
}
INVALIDATE 的流程如下：
这些流程在图像消费者都有详细讲过，我们直接看 handleMessageInvalidate 函数的实现
/frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp
bool SurfaceFlinger::handleMessageInvalidate() {
return handlePageFlip();
}
bool SurfaceFlinger::handlePageFlip()
{
bool visibleRegions = false;
bool frameQueued = false;
bool newDataLatched = false;
……
for (auto& layer : mLayersWithQueuedFrames) {
//遍历所有Layer，获取缓冲区
const Region dirty(layer->latchBuffer(visibleRegions, latchTime));
layer->useSurfaceDamage();
invalidateLayerStack(layer->getLayerStack(), dirty);
if (!dirty.isEmpty()) {
newDataLatched = true;
}
}
mVisibleRegionsDirty |= visibleRegions;
if (frameQueued && mLayersWithQueuedFrames.empty()) {
signalLayerUpdate();
}
return !mLayersWithQueuedFrames.empty() && newDataLatched;
}
handlePageFlip 函数中调用 layer->latchBuffer() 函数来获取缓冲区
/frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp
Region Layer::latchBuffer(bool& recomputeVisibleRegions, nsecs_t latchTime)
{
status_t updateResult = mSurfaceFlingerConsumer->updateTexImage(&r,
mFlinger->mPrimaryDispSync, &mAutoRefresh, &queuedBuffer,
mLastFrameNumberReceived);
……
return outDirtyRegion;
}
latchBuffer 函数非常的长，流程也很多，但我们只关心它获取图像缓冲区的部分：调用 SurfaceFlingerConsumer 的 updateTexImage 函数。SurfaceFlingerConsumer 是对 BufferQueueConsumer 的封装，我们看看 updateTexImage 的实现。
/frameworks/native/services/surfaceflinger/SurfaceFlingerConsumer.cpp
status_t SurfaceFlingerConsumer::updateTexImage(BufferRejecter* rejecter,
const DispSync& dispSync, bool* autoRefresh, bool* queuedBuffer,
uint64_t maxFrameNumber)
{
BufferItem item;
//获取缓冲区
err = acquireBufferLocked(&item, computeExpectedPresent(dispSync),
maxFrameNumber);
……
// 释放前一个缓冲区
#ifdef USE_HWC2
err = updateAndReleaseLocked(item, &mPendingRelease);
#else
err = updateAndReleaseLocked(item);
#endif
if (err != NO_ERROR) {
return err;
}
……
return err;
}
updateTexImage 函数中主要做了两件事情：
下面分别看一下这两件事情的实现。
acquireBufferLocked 函数的实现如下：
/frameworks/native/services/surfaceflinger/SurfaceFlingerConsumer.cpp
status_t SurfaceFlingerConsumer::acquireBufferLocked(BufferItem* item,
nsecs_t presentWhen, uint64_t maxFrameNumber) {
status_t result = GLConsumer::acquireBufferLocked(item, presentWhen,
maxFrameNumber);
if (result == NO_ERROR) {
mTransformToDisplayInverse = item->mTransformToDisplayInverse;
mSurfaceDamage = item->mSurfaceDamage;
}
return result;
}
这里调用了 GLConsumer 的 acquireBufferLocked，SurfaceFlingerConsumer 是继承自 GLConsumer，所以其实是调用父类方法。
/frameworks/native/libs/gui/ConsumerBase.cpp
status_t ConsumerBase::acquireBufferLocked(BufferItem *item,
nsecs_t presentWhen, uint64_t maxFrameNumber) {
if (mAbandoned) {
CB_LOGE("acquireBufferLocked: ConsumerBase is abandoned!");
return NO_INIT;
}
//获取缓冲区
status_t err = mConsumer->acquireBuffer(item, presentWhen, maxFrameNumber);
if (err != NO_ERROR) {
return err;
}
if (item->mGraphicBuffer != NULL) {
if (mSlots[item->mSlot].mGraphicBuffer != NULL) {
freeBufferLocked(item->mSlot);
}
mSlots[item->mSlot].mGraphicBuffer = item->mGraphicBuffer;
}
mSlots[item->mSlot].mFrameNumber = item->mFrameNumber;
mSlots[item->mSlot].mFence = item->mFence;
return OK;
}
releaseBufferLocked 函数的实现如下
/frameworks/native/libs/gui/ConsumerBase.cpp
status_t ConsumerBase::releaseBufferLocked(
int slot, const sp<GraphicBuffer> graphicBuffer,
EGLDisplay display, EGLSyncKHR eglFence) {
if (mAbandoned) {
CB_LOGE("releaseBufferLocked: ConsumerBase is abandoned!");
return NO_INIT;
}
//释放缓冲区
status_t err = mConsumer->releaseBuffer(slot, mSlots[slot].mFrameNumber,
display, eglFence, mSlots[slot].mFence);
if (err == IGraphicBufferConsumer::STALE_BUFFER_SLOT) {
freeBufferLocked(slot);
}
mPrevFinalReleaseFence = mSlots[slot].mFence;
mSlots[slot].mFence = Fence::NO_FENCE;
return err;
}
可以看到 acquireBufferLocked 和 releaseBufferLocked 最终调用的依然是 BufferQueueConsumer 的 acuqireBuffer 和 releaseBuffer 函数。
到这里，图像显示原理的知识就都讲完了。技术飞速发展，Android 中不断有新的图像显示系统出现，如 RN，小程序，以及现在很火的Flutter 等等，当面对层出不穷的新技术时，我们只有掌握底层的本质原理和组成，才能快速的理解新的技术。