Monthly Archives: August 2019

Laravel Copy Tables To Other Server

Recently in one of my Laravel project, I had a situation where some of the tables are required to copy from one db server to other db server on real time basis.

Laravel Observer
Laravel

First of all I thought about using binary logging but it, but my hosting provider refused to enable binary logging. Other option is to use federated storage engine. But hosting provider refused to enable it too. In fact the tech support team asked me a question that what is Fedarated storage engine? So I gave up the hope on them. Finally I implemented with the strategy that I am going to explain in this blog.

Using Laravel Observer and Eloquent

In Laravel we can use model observers to listen to the events on models and take actions. Here are steps you have to perform.

Step 1 : Create Observer

Create a new Class in App\Http\Observers folder and name is ModelObserver

Step 2 : Assign Observer to Model

In your model file add boot function as following.

public static function boot()
{
parent::boot();
YourModelName::observe(new \App\Http\Observers\ModelObserver);
}

Step 3 : Add Methods to Your Observer

In your ModelObserver class add following methods.

public function created($model)
{

}

public function updated($model)
{

}

Now when you insert or update record with Eloquent it will invoke created and updated method and will send model instance as param.

Replication to Other Server

Now next step is to implement replication strategy. For this we are going to create separate model and use remote server connection with that model.

Step 1 : Save Remote Host Connection

For this first of all create saparate MySql connection in your config/database.php file

‘replica’ => [
‘driver’ => ‘mysql’,
‘host’ => env(‘REPLICA_DB_HOST’, ‘remotehost’),
‘port’ => env(‘REPLICA_DB_PORT’, ‘3306’),
‘database’ => env(‘REPLICA_DB_NAME’, ‘your_db’),
‘username’ => env(‘REPLICA_DB_USERNAME’, ‘your_username’),
‘password’ => env(‘REPLICA_DB_PASSWORD’, ‘your_password’),
‘charset’ => ‘utf8’,
‘collation’ => ‘utf8_unicode_ci’,
‘prefix’ => ”,
‘strict’ => false,
‘engine’ => null,
]

Step 2 : Create Model to Connect to Remote Host

Now create a separate model and assign remote server connection to it.

<?php

namespace App\Http\Models;

use Illuminate\Database\Eloquent\Model;

class RemoteModel extends Model
{
protected $connection= ‘replica’;
protected $table = ‘remote_table’;
}

Step 3 : Replicate Model

Now inside your observer’s created method add following code.

if($model instanceOf \App\Http\Models\Model){
$modelArray = $model->toArray();
$newModel = new \App\Http\Models\RemoteModel;
foreach( $modelArray as $key => $value ){
if(gettype($modelArray[$key]) != ‘array’){
$newModel[$key] = $modelArray[$key];
}
}
$newModel->save();
}

As you can see in above code that we are checking first if model created is type of our Model Class and then creating object of RemoteModel and assigning values of Model to RemoteModel object and then finally call save method, this will create row in remote table on remote server.

Access Amazon RDS from AWS Lambda

Recently I was working with serverless application created in NodeJs which was deployed to AWS Lambda. This app is connecting to MySQL database which was deployed to RDS. While I was testing in local machine it worked very well.

AWS Lambda Amazon RDS

After deployed it stopped working. After struggling for half a day I was able to finally make it working. In this blog I am going to mention what it takes to make it working.

Different Security Group is the First Solution

This is really strange, if your Amazon RDS and AWS Lambda function shares the security group it will never connect. Not sure why it’s this way, but to solve the problem.

  1. Create New Security Group and assign it to your Lambda function.
  2. Create Other Security Group and assign it your RDS
  3. Allow MySQL connection in RDS Security Group

To allow MySQL connection, add inbound rules. Select MYSQL/Aurora fromType and allow it from 0.0.0.0/0

Allow VPC Access to Lambda Function

  1. Go to Roles page in IAM consle.
  2. Create new role and add policy AWSLambdaVPCAccessExecutionRole
  3. Assign this role to your Lambda function

Use Connection Pool in Your Code

While using serverless you have to be careful about asynchronous operations. Connection to MySQL is also asynchronous operations. If you do not handle it properly, lambda function will give you timeout as it’s expecting callback function to return result but because it’s async operations it will take time. So you have to explicitly let it know that, there is async operation so wait for it.

So for that create connection pool.

var mysql = require(‘mysql’);
var pool = mysql.createPool({
user : ‘username’,
password : “password”,
database: ‘database’,
host : ‘rds_end_point’
});

Now wait to get connection from the pool.

pool.getConnection((err, connection) => {
connection.query(‘SQL’,function(err, rows, fields) {
const response = {
statusCode: 200,
headers: {
“Access-Control-Allow-Origin” : “*” // Required for CORS support to work
},
body: JSON.stringify({
success: true,
rows: rows
})
};
connection.release();
callback(null, response);
});
})

As you can see in above code we are waiting for getting connection. Once we have the connection we query the database and get the result. After that we invoke the callback and release the connection so it can be used for next requests.