Category Archives: Database

Observer, Quorum

Leave a reply

This article closes the series for DG and Fast-Start Failover that I covered with more details the case of isolation can leverage the shutdown of your healthy/running primary database. The “ORA-16830: primary isolated from fast-start failover partners”.

In the first article, I wrote about how one simple detail that impacts dramatically the reliability of your MAA environment. Where you put your Observer in DG environment (when Fast-Start Failover is in use) have a core figure in case of outages, and you can face Primary isolation and shutdown. Besides that, there is no clear documentation to base yourself on “pros and cons” to define the correct place for Observer. You read more in my article here.

In the second article, I wrote about one new feature that can help to have more protected and cover more scenarios for Fast-Start Failover/DG. Using Multiple Observers you can remove the single point of failure and allow you to put one Observer in each side of your environment (primary, standby, and a third one). You can read more in my article here.

In this last article, I discuss how, even using all the features, there is no perfect solution. Another point is discussing here is how (maybe) Oracle can improve that. Below I will show more details that even multiple observers continue to shutdown a healthy primary database. Unfortunately, it is a lot of tech info and is a log thread output. But you can jump directly to the end to see the discussion about how this can be improved.

More…

ODA, ACFS and ASM Dilemma

1 Reply

As you know, for ODA, you have two options for storage: ACFS or ASM. If you choose ACFS, you can create all versions for databases, from 11g to 18c (until this moment). But if you choose ASM, the 11g will not be compatible.

So, ASM or ACFS? If you choose ACFS, the diskgroup where ACFS runs will be sliced and you have one mount point for each database. If you have, as an example, one system with more than 30 databases, can be complicated to manage all the ACFS mount points. So, ASM it simple and easier solution to sustain. Besides the fact that it is more homogeneous with other database environments (Exadata, RAC’s …etc).

If you choose ASM you can’t use 11g versions or avoid the ACFS mount points for all databases, but you can do a little simple approach to use 11g databases and still use ASM for others. Took one example where just 3 or 4 databases will run over 11g version and all others 30 databases in the environment will be in 12/18. To achieve that, the option, in this case, is using a “manual” ACFS mount point, I will explain.

More…

ODA, JSON and FLASH

1 Reply

Recently, in March, I made the reimage from an X5-2 HA ODA and saw a strange behavior during the diskgroup creation and couldn’t reproduce (because involve reimaging again). Basically, the FLASH diskgroup was not created.

But in last May I reimaged another ODA using the same patch/imageversion (18.3.0.0 – Patch 27604623) and was possible to verify again. In both cases, I created the appliance using the CLI “odacli create-appliance” using JSON file because the network uses VLAN (what it is impossible to create using the web interface), and both appliances are identical (X5-2 HA with SSD for RECO and FLASH).

To reimage, I followed the steps in the docs for this version and used the ISO to do the baremetal procedure. If you look in the docs about the options for storage (check here) you can see that there is no single reference to use FLASH diskgroup (or that you need to do that). Checking in the readme/reference JSON files that exist in the folder “/opt/oracle/dcs/sample” under file “sample-oda-ha-json-readme.txt”:

More…

Observer, More Than One

2 Replies

Recently I made a post about a little issue that I got with Oracle Data Guard. In that scenario, because of outage in the standby datacenter, healthy primary database shutdown with error “ORA-16830: primary isolated…”. Just to remember that the database was running with Maximum Availability, Fail-Start Failover enabled, and (the most important detail) the Observer was running in the standby datacenter too.

The point from my previous post tried to show that does not exists one doc that provides full details about “pros” and “cons” where put your observer. Whatever the place, on the primary datacenter or in standby, it has little details to check. Even the best (ideal) scenario, with a third datacenter, can be tough to sustain.

Here I will try to show one option that can help you and improve the reliability of your MAA/DG environment. At least, you will have more options to decide how to protect your database. Bellow, I show some details about how to configure and use multiple observers, but if you want to jump and see a little concern you can directly to the end of the post.

More…

DML over Standby for Active Data Guard in 19c

2 Replies

With the new 19c version the Data Guard received some attention and now we can do DML over the standby and it will be redirect to primary database. It is not hard to implement, but unfortunately there is no much information about that in the docs about that.

As training exercise I tested this new feature and want to share some information about that. First, the environment that I used (and the requirements too):

  • Primary and Standby databases running 19c.
  • Data Guard in Maximum Availability .
  • Active Data Guard enabled.

Remember that the idea of DML over the standby it is to use in some cases where your reporting application need to update some tables and few records (like audit logins) while processing the data in the standby. The volume of DML is (and will be) low. At this point there is no effort to allow, or create, a multiple active-active datacenters/sites for your database. If you start to execute a lot of DML in the standby side you can impact the primary database and you adding the fact that you can maximize the problems for locks and concurrency.

More…

Exadata X8, Second look

Leave a reply

Every year Oracle arrives and release new version of Exadata with a plenty of new things. We have the natural evolution from hardware (more memory, more cpu…) and sometimes news features from software side. The point for this post today it is not about the HW and SW things, but something is hidden in the small lines of the new X8.

More…

Observer, Where?

6 Replies

Some months ago I got one error with Oracle Data Guard and now I had time to review it again and write this article. Just to be clear since the beginning, the discussion here is not about the error itself, but about the circumstances that generated it.

The environment described here follows, at least, the most common best practices for DG by Oracle. Have 1 dedicated server for each one: Primary Database, Physical Standby Database, and Observer. The primary and standby reside in different data centers in different cities, dedicated network for interconnecting between sites, protection mode was Maximum Availability, and runs with Fast-Start Failover enabled (with 30 seconds for threshold). The version here is 12.2 but will be the same for 19c. So, nothing so bad in the environment, basic DG configuration trying to follow the best practices.

More…

Reaching Exadata 18c

3 Replies

Here I cover in raw, undocumented and uncommented mode the process to update and upgrade your Exadata using the last version of everything. AND since Oracle 18c was released to use with Oracle Exadata (from SQL Maria), this post include the Oracle 18c upgrade for Grid Infrastructure and Oracle database binaries installation.

Since one friend was decommissioning one old Exadata X2 (running after the end of life), I used to do some tests and here you will find the commands, outputs, images that I use to (in order):

  • Apply the patch 12.1.0.2.180116 for Oracle Grid Infrastructure 12.1.
  • Update the Infiniband switches to last version 2.2.7-1.
  • Apply the patch for Exadata Storage Server using the last version, 18.1.4.0.0.180125.3.
  • Apply the patch for Exadata Database Servers using the last version, 18.1.4.0.0.180125.3
  • Upgrade the Oracle Grid Infrastructure 12.1 to Grid Infrastructure 12.2, applying PSU 12.2.0.1.180116 at same time.
  • Upgrade the Oracle Grid Infrastructure 12.2 to Oracle Grid Infrastructure 18c.
  • Install Oracle database binaries for Oracle Database 18c and Create a test Oracle Rac Instance.

Continue lendo…

To infinity and beyond

Leave a reply

First, I need to sorry about the long delay since my last update. The last one was in 03/August/2015, so, more than 2 years without news. But I can’t complain about that, was a really busy time here:

  • Oracle Zero Data Loss Recovery Appliance (ZDLRA) deployment: two of them in the end of 2015 and begin of 2016 and working together with two Oracle Exadata to build DR site and reaching RPO and RTO zero. First ZDLRA of Brazil, second in American Latina and one of the first of world to achieve this kind of level.
  • Oracle Open World: linked with the first topic. Because of the size of the project and high complexity of the project I was speaker (together with ZDRLA dev team) in Oracle Open World SFO 2015. Link for presentation here.
  • DR site: specify/implement/deploy/etc DR site during 2015 and 2016, EXA+EXA+DG+ZDLRA+RPO+RTO. So, busy work that unfortunately is not plug and play.
  • Exadata: New deployments to achieve more throughput/performance

For life:

  • Big Move: at the end of 2017 I moved from Brazil to work with a new challenge as DBA in Europe, Luxembourg at eProseed (http://www.eproseed.com).
  • Family: Family increased, basically 1+1=3 (and this count is correct, don’t worry).

I promise to update more constantly here. Stay tuned!

Aplicando Patch no Exadata

3 Replies

O appliance Oracle Exadata é uma das tecnologias mais modernas para banco de dados Oracle, a união de Hardware e Software. Mas este software precisa ser atualizado de tempos em tempos você precisa aplicar o patch.

O time de Engineered Systems da Oracle disponibiliza os updates para serem aplicadas em toda a pilha: do Exadata Software aos binários do banco de dados. É tarefa do DMA acompanhar isso e deixar seu ambiente atualizado, muitas vezes (aqui no Brasil) estes updates são aplicados pelo time de ACS da Oracle, mas nada impede que você mesmo faça isso.

Este é o foco destes artigos, vou mostrar como proceder com um update completo do Oracle Exadata: Exadata Software, Infiniband, Linux, Update do Grid 11GR2 para 12C e Aplicação de BP nos binários do banco. Eu já descrevi sobre isso no meu blog a uns 3 anos atrás (aqui e aqui), mas muita coisa mudou desde aquela época.

Planejamento

De qualquer forma, antes de começar qualquer update (Oracle Exadata ou não) temos que planejar, verificar qual a versão que estamos e qual queremos aplicar.

Para o Oracle Exadata temos uma nota que contempla tudo isso e que é o início de qualquer update: Exadata Database Machine and Exadata Storage Server Supported Versions (Doc ID 888828.1). Nesta nota estão todas as informações importantes como documentação, as versões existentes, versões recomendadas e matriz de compatibilidade entre Exadata Software e binários do banco.

Então, o primeiro passo é o planejamento, ler e compreender esta nota, principalmente com as versões disponíveis e quais serão aplicadas. Outro ponto fundamental do planejamento é conhecer o seu ambiente, no meu caso a imagem que tenho aqui é a 11.2.3.3.0.

No tópico Exadata Storage Server 12c da nota identificamos que a última versão e que será aplicada aqui é a 12.1.2.1.2 (image version 12.1.2.1.2.150617.1) e composta pelos seguintes patches e docs:

  • Patch 20748218 – Storage server e InfiniBand
  • Patch 21151982 – Database server (para quem não utiliza OVM)
  • Readme – Note 2014306.1

Na mesma nota temos as versões do Grid Infrastructure e dos bancos de dados que são compatíveis com as imagens do Exadata. Como o meu planejamento inclui atualizar tudo para a última versão estes também serão atualizados.

Através da nota os seguintes updates serão realizados (e demonstrados aqui nestes artigos):

  • Update da imagem 11.2.3.3.0 para a 12.1.2.1.2 do Exadata (storage e bancos)
  • Upgrade da versão 11.2.0.4 BP 6 do Grid para a versão 12.1.0.2 BP 9
  • Update da versão 11.2.0.4 BP6 do banco de dados para a versão 11.2.0.4 BP 16

Pensando no appliance Oracle Exadata e na relação entre seus componentes o primeiro ponto a ser atualizado é a própria imagem do Exadata Storage Server (Linux, Exadata Software). Depois temos switchs Infiniband. Na sequência temos o update dos Database Servers e seus componentes (Grid e os binários do banco de dados).

Antes de iniciar o update precisamos montar o plano de atualização, qual a ordem de instalação dos patches. Isso só pode ser feito lendo todos os Readmes dos patches que vamos aplicar. Lendo o Readme do Patch 20748218 e Patch 21151982 descobrimos que existem dois métodos possíveis para atualizar o Database Server.

O primeiro é através da imagem ISO que acompanha o Patch 21151982 e o segundo é através da criação do repositório Exadata local. Este último método é o que utilizarei aqui, ele tem algumas vantagens que descreverei em detalhes depois mas é importante entender como funciona. Basicamente você cria um repositório local com os rpm’s que serão aplicados (os mesmo presentes na ISO). Infelizmente esta criação pode demorar um pouco e por isso será o primeiro procedimento a ser realizado.

Através da leitura de todas as notas do MOS envolvidas e de todos os Readmes dos patches aplicados o seguinte plano foi criado:

  1. Criação do repositório Exadata para Database Server
  2. Update do Storage Server e Switch Infiniband
  3. Update do Database Server
  4. Upgrade do GRID para 12C e Instalação dos binários 12C
  5. Update dos binários 11.2.0.4 para o BP16

Abaixo a descrição detalhas de cada uma destas etapas acima listadas.

A última atualização deste artigo foi em 03/08/2015.